Web CNLearn - Vue 4 (Post Number 4 That Is) - Connecting to Backend, Performing Actual Search, Displaying Results, Testing
Today we will understand our backend and continue working on the frontend.
(Like I said in the previous post, at some point, I want to create a static version of this website -> just search functionality and nothing else and host it on GH Pages. Inspired by this. Basically saving this here as a reminder for myself.)
Backend API Endpoints
Let’s connect to our (temporary) backend. I have a working FastAPI backend that simply returns words and characters definitions for now. It is slightly different to the one we are building in the FastAPI series (post 1 and post 2). In what sense? Well, it only returns the words and characters. The new version I am building will have a lot more functionality. BUT, the words/characters it returns will be the same. There are two endpoints our frontend will call:
API/words/simplified/{word}
API/characters/simplified/{character}
Let’s call the first one getWord and the second one getCharacter.They are both GET requests.
getWord API Endpoint
Let’s see what getWord returns:
[
{
"id": 0,
"simplified": "string",
"traditional": "string",
"pinyin_num": "string",
"pinyin_accent": "string",
"pinyin_clean": "string",
"pinyin_no_spaces": "string",
"also_written": "string",
"classifiers": "string",
"definitions": "string",
"frequency": 0
}
]
For example, if I send in the word “好”, I get:
[
{
"id": 27949,
"simplified": "好",
"traditional": "好",
"pinyin_num": "hao3",
"pinyin_accent": "hǎo",
"pinyin_clean": "hao",
"pinyin_no_spaces": "hao",
"also_written": "",
"classifiers": "",
"definitions": "good; well; proper; good to; easy to; very; so; (suffix indicating completion or readiness); (of two people) close; on intimate terms; (after a personal pronoun) hello",
"frequency": 165789
},
{
"id": 27950,
"simplified": "好",
"traditional": "好",
"pinyin_num": "hao4",
"pinyin_accent": "hào",
"pinyin_clean": "hao",
"pinyin_no_spaces": "hao",
"also_written": "",
"classifiers": "",
"definitions": "to be fond of; to have a tendency to; to be prone to",
"frequency": 165789
}
]
So it’s a list of objects. What if the word doesn’t exist? For example, if I getWord(好闷), I get the empty list []. So our frontend has to be able to deal with getting multiple results, one result or zero results. Your next question might be: Why would it returns zero results? Well, imagine that our segmenter makes a mistake. And it requests a word that is not a word. That is a problem (opportunity??) in Chinese to be honest. Segmentation is difficult. So what should our application do then? Let’s think of the flow.
We enter the phrase 我们是谁. The segmenter (and it actually does) splits these into “我们” and “是谁”. The second string is not a word. We hit the endpoint and…get an empty list. The application will then go character by character and return that. So in the end, we will have “我们”, “是” and “谁”. Ok we’ll code that. But first, let’s also look at the other API endpoint.
getCharacter API Endpoint
This is what it returns:
{
"id": 0,
"character": "string",
"definition": "string",
"pinyin": "string",
"decomposition": "string",
"etymology": {},
"radical": "string",
"matches": "string",
"frequency": 0
}
For example, if I do getCharacter(好), we get:
{
"id": 1595,
"character": "好",
"definition": "good, excellent, fine; proper, suitable; well",
"pinyin": "hǎo",
"decomposition": "⿰女子",
"etymology": {
"type": "ideographic",
"hint": "A woman 女 with a son 子"
},
"radical": "女",
"matches": "[[0], [0], [0], [1], [1], [1]]",
"frequency": 165789
}
Dictionary Service
Ok let’s write something that will allow us to call the API endpoints. I’ll use Axios for that so let’s install it first.
npm install axios
Then we create a services directory in our src folder and add a Dictionary.js file in there.
import axios from 'axios';
const fastApiClient = axios.create({
baseURL: 'http://localhost:8000',
withCredentials: false,
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
});
export default {
getWord(simplified) {
return fastApiClient.get(`/words/simplified/${simplified}`);
},
getCharacter(character) {
return fastApiClient.get(`/characters/simplified/${character}`);
},
};
s
We’re not gonna test these. We’re gonna assume the backend team (i.e. me) tested the backend, and they/I did. Ok let’s go back to our Vuex search module and develop our search function.
Developing our Search Implementation
At the moment, when we enter a string in the SearchInput it gets segmented and the segmented words get displayed. Let’s check that the segmented words actually “exist” (in the database) and if not, split the non existing words into characters. First, let’s import our Dictionary service.
import Dictionary from '@/services/Dictionary';
...
Then our findWords action will look something like this:
findWords({ state, commit, dispatch }, words) {
console.log("hi");
}
It will have access to the state, to commit (i.e. be able to commit mutations) and to dispatch (be able to dispatch other actions). It also receives an argument of words. Why do that though?
Why not take the word from the segmentedWords state? Let’s write some pseudocode. Imagine our current segmentedWords state contains the value “我们” for key/position 0 and the value “是谁” for key/position 1. Remember that the key in our map is the position and the value is the word.
# everything is taken from state
findWords():
for word in segmentedWords.values:
result = Dictionary.getWord(word)
if result != []: # a word was found
add word to foundWords array
else:
# a word was not found. let's split by character
# but then, what would we do?
# should we then do:
for character in word:
result = Dictionary.getWord(word)
# what if somehow something is not found?
# do we add yet another if statement?
OK that is messy. I don’t like it. What will we do instead? Well the findWords action will actually receive a words argument. It will then go word by word and if the word is found, adds it to foundWords. If it doesn’t, it recalls itself (recursively) and when it does that, it sends a new list of words (i.e. the characters that the unknown word was split into).
Let’s implement that. Let’s console.log() everything first, not creating any wordsFound state just yet.
findWords({ dispatch }, words) {
words.forEach((word) => {
Dictionary.getWord(word)
.then((response) => {
if (!(response.data.length === 0)) {
const wordObject = {
simplified: word,
data: response.data,
};
console.log(wordObject);
} else {
dispatch('findWords', [...word]);
}
}).catch((error) => console.log(error));
});
},
Ok so we have a recursive function that works. Now let’s do something with the data we get. Let’s create some state. We will actually create two things: a cachedWords map and a currentWords set. The currentWords set will contain the simplified words that are found. The cachedWords map will contain the actual data. Why am I doing this? Because if the user then searches for the same word again, there’s no need to hit our API endpoint.
New State and Mutations and Updating fomd
So what do we have in our state now?
state: () => ({
searchString: '',
segmentedWords: new Map(),
cachedWords: new Map(),
currentWords: new Set(),
}),
What mutations do we need? We need an ADD_TO_CACHE mutation and an AD_TO_CURRENT_WORDS mutation. And let’s also have a CLEAR_CURRENT_WORDS.
ADD_TO_CACHE(state, wordObject) {
const { simplified, data } = wordObject;
state.cachedWords.set(simplified, data);
},
ADD_TO_CURRENT_WORDS(state, simplified) {
state.currentWords.add(simplified);
},
CLEAR_CURRENT_WORDS(state) {
state.cachedWords.clear();
},
Then, let’s add some new actions:
addToCache({ commit }, wordObject) {
commit('ADD_TO_CACHE', wordObject);
},
addToCurrentWords({ commit }, simplified) {
commit('ADD_TO_CURRENT_WORDS', simplified);
},
You might wonder. Why have separate actions just for the mutation? Well, I will probably need to call them separately at some point. So I’ll have that there. Then, our search action becomes:
search({ dispatch }, searchString) {
commit('CLEAR_CURRENT_WORDS');
const chineseString = extractChineseCharactersToString(searchString);
dispatch('setSearch', chineseString);
// segment the words
const segmentedWordsArray = segmentit.doSegment(chineseString, {
simple: true,
stripPunctuation: true,
});
dispatch('addToSegmentedWords', segmentedWordsArray);
// then do the actual word search
dispatch('findWords', segmentedWordsArray);
},
And finally, let’s make a modification to our findWords action.
findWords({ dispatch }, words) {
words.forEach((word) => {
Dictionary.getWord(word)
.then((response) => {
if (!(response.data.length === 0)) {
const wordObject = {
simplified: word,
data: response.data,
};
// add it to the character cache
dispatch('addToCache', wordObject);
// add to current words
dispatch('addToCurrentWords', wordObject.simplified);
} else {
dispatch('findWords', [...word]);
}
}).catch((error) => console.log(error));
});
},
Ok so we get a search string. We clear current words set, we extract the chinese characters and set the search string. We segment it. We then find the words in the segmented words. That’s when the findWords action is dispatches. Ok all good so far. But what does our findWords action currently do? It receives an array of words. Then for each of the words it calls the API. If it gets some data, it adds it to the cache and to current words. If not, splits it into characters and recursively dispatches the findWords action again.
Still, we are writing to the cache and not doing anything with it. In fact if you enter the same search string a few times in a row you’ll request it again and again. Let’s be nice to our server. Let’s add a nice if/else condition somewhere in there.
findWords({ dispatch, state }, words) {
words.forEach((word) => {
if (!state.cachedWords.has(word)) {
Dictionary.getWord(word)
.then((response) => {
if (!(response.data.length === 0)) {
const wordObject = {
simplified: word,
data: response.data,
};
// add it to the character cache
dispatch('addToCache', wordObject);
// add to current words
dispatch('addToCurrentWords', wordObject.simplified);
} else {
dispatch('findWords', [...word]);
}
}).catch((error) => console.log(error));
} else {
// add it to current words
dispatch('addToCurrentWords', word);
}
});
},
Ok so now it first checks if it’s in the cache. If it is, it doesn’t request it from the server again. This is useful when in a a session the user might enter a word that’s already been searched for before.
What is the issue we run into now? Well, the axios promises are not necessarily resolved in order. If you search for a phrase, you might get a different order for the results than what you expected. Here I was not too certain what the best way. My original implementation actually involved a function that would look for where the words were in the original searchString but now I actually want to try something new. :) So let’s use await inside a loop. I want them to be resolved sequentially.
So let’s rewrite our findWords action and make it async:
async findWords({ dispatch, state }, words) {
/* eslint-disable no-restricted-syntax */
for (const word of words) {
if (!state.cachedWords.has(word)) {
try {
/* eslint-disable no-await-in-loop */
const response = await Dictionary.getWord(word);
if (!(response.data.length === 0)) {
const wordObject = {
simplified: word,
data: response.data,
};
dispatch('addToCache', wordObject);
dispatch('addToCurrentWords', wordObject.simplified);
} else {
await dispatch('findWords', [...word]);
}
} catch (error) {
console.log(error);
}
} else {
dispatch('addToCurrentWords', word);
}
}
},
Please note also having to ask eslint to not warn us about it. We know why we’re doing it. Now, if you you enter a search string, it will return it to you in order. :) Yay! Ok, now let’s change the SearchResults component to show our currentWords instead of segmentedWords.
<template>
<div data-test="searchResultsDiv" class="search_results">
<div
data-test="searchResultsFound"
v-if="currentWords.size>0"
class="container p-grid p-flex-column"
>
<p v-for="word of currentWords" :key="word">{{ word }}</p>
</div>
<div data-test="searchResultsBlank" v-else>
<p>Enter some text to get started.</p>
</div>
</div>
</template>
<script>
import { mapState } from 'vuex';
export default {
computed: {
...mapState('search', ['currentWords']),
},
};
</script>
<style scoped>
.container {
display: flex;
flex-direction: column;
align-items: center;
}
</style>
We’ve done lots today. Are we finally done? Of course not. Let’s test.
Testing
We will learn some new concepts today. Keep in mind that now our Vuex store is hitting our backend API so we’ll have to mock those calls. YAY for learning…
If we run our npm run test:unit now, it will fail. Well, some of them at least. SearchInput tests are fine. We don’t care about the failed GET requests there. SearchResults, however, we do care. Actually, we don’t care. We are still just testing the component. Not the Vuex store. Consequently, let’s just rewrite some tests and set the state directly. In the next post, literally after this one, we will have a look at testing the Vuex store -> separately first -> then in our components so far.
We need to change both ‘SearchInput.spec.js’ and ‘SearchResults.spec.js’. The reason we have to change both is that we now have some async functions in the store so when we use the factory function, we need to await that.
Also, I made some styling changes (I keep using Python conventions….).
import { mount } from '@vue/test-utils';
import SearchInput from '@/components/SearchInput.vue';
import PrimeVue from 'primevue/config';
import search from '@/store/modules/search';
import { createStore } from 'vuex';
const factory = async () => {
const store = await createStore({
modules: {
search,
},
});
const wrapper = await mount(SearchInput, {
global: {
plugins: [PrimeVue, store],
},
});
return { store, wrapper };
};
describe('SearchInput', () => {
test('Checks if inputting values works', async () => {
const { store, wrapper } = await factory();
// first check that the searchString in the search module state is null
expect(store.state.search.searchString).toBe('');
// set the searchString data using setData method
await wrapper.setData({ localSearchString: '你好' });
expect(wrapper.find('[data-test="searchString"]').exists()).toBe(true);
const searchString = wrapper.find(
'[data-test="searchString"]',
).element.textContent;
expect(searchString).toBe('你好');
// let's check the store state as well (obviously should pass if above passes)
expect(store.state.search.searchString).toBe('你好');
// let's also "write some text" in the textfield using setValue
await wrapper.find('[data-test="searchBox"]').setValue('我们的朋友');
const updatedSearchString = wrapper.find('[data-test="searchString"]').element.textContent;
expect(updatedSearchString).toBe('我们的朋友');
// let's check the store state as well (obviously should pass if above passes)
expect(store.state.search.searchString).toBe('我们的朋友');
});
test('Checks if the component got mounted', async () => {
const { store, wrapper } = await factory();
expect(wrapper.find('[data-test="searchedFor"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchBox"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchString"]').exists()).toBe(false);
// let's also check that the searchString in the search module state is null
expect(store.state.search.searchString).toBe('');
});
test('Checks that the search value is blank in a new test', async () => {
const { store } = await factory();
// first check that the searchString in the search module state is null
expect(store.state.search.searchString).toBe('');
});
test('Checks if anything that is not a Chinese character gets filtered out', async () => {
const { store, wrapper } = await factory();
// first check that the searchString in the search module state is null
expect(store.state.search.searchString).toBe('');
// set the searchString data using setData method
await wrapper.setData({ localSearchString: '你们是our friends' });
expect(wrapper.find('[data-test="searchString"]').exists()).toBe(true);
const searchString = wrapper.find(
'[data-test="searchString"]',
).element.textContent;
expect(searchString).toBe('你们是');
// let's check the store state as well (obviously should pass if above passes)
expect(store.state.search.searchString).toBe('你们是');
// let's also "write some text" in the textfield using setValue
await wrapper.find('[data-test="searchBox"]').setValue('我们的朋友are you');
const updatedSearchString = wrapper.find('[data-test="searchString"]').element.textContent;
expect(updatedSearchString).toBe('我们的朋友');
// let's check the store state as well (obviously should pass if above passes)
expect(store.state.search.searchString).toBe('我们的朋友');
});
});
And then for SearchResults.spec.js, I made a few changes. Since we use the currentWords set to display the results instead of segmentedWords, we need to set that in the state.
import { mount } from '@vue/test-utils';
import SearchResults from '@/components/SearchResults.vue';
import PrimeVue from 'primevue/config';
import search from '@/store/modules/search';
import { createStore } from 'vuex';
import { nextTick } from 'vue';
import { zip } from 'lodash';
const factory = async () => {
const store = await createStore({
modules: {
search,
},
});
const wrapper = await mount(SearchResults, {
global: {
plugins: [PrimeVue, store],
},
});
return { store, wrapper };
};
describe('SearchResults', () => {
test('Checks if the component got mounted', async () => {
const { store, wrapper } = await factory();
expect(wrapper.find('[data-test="searchResultsDiv"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchResultsFound"]').exists()).toBe(false);
expect(wrapper.find('[data-test="searchResultsBlank"]').exists()).toBe(true);
// let's also check that the currentWords is also blank
expect(store.state.search.currentWords.size).toEqual(0);
});
test('Checks if currentWords are shown', async () => {
const { store, wrapper } = await factory();
expect(wrapper.find('[data-test="searchResultsDiv"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchResultsFound"]').exists()).toBe(false);
expect(wrapper.find('[data-test="searchResultsBlank"]').exists()).toBe(true);
const currentWordsArray = ['我们', '是', '你们', '的', '朋友'];
currentWordsArray.forEach((word) => {
store.state.search.currentWords.add(word);
});
await nextTick();
expect(wrapper.find('[data-test="searchResultsDiv"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchResultsFound"]').exists()).toBe(true);
expect(wrapper.find('[data-test="searchResultsBlank"]').exists()).toBe(false);
// let's see if we can see the 5 <p></p> items
const pElements = wrapper.findAll('p');
// now let's iterate over each pElement and each word in segmentedWordsArray
// let's zip them together...yes Python I know
const pElsAndWords = zip(pElements, currentWordsArray);
pElsAndWords.forEach((pElementAndWord) => {
const pElement = pElementAndWord[0];
const word = pElementAndWord[1];
// now check the text in each one and compare with the word in the array
expect(pElement.text()).toEqual(word);
});
});
});
We run npm run test:unit, all good, and that is all for today…In the post I’m writing now we’ll test our Vuex store. In the one after and then we’ll test our components calling the Vuex store. In the one after we’ll start making our search results pretty. In the one after we will make a Word view component. Plenty to write about!
(Also, I am aware I haven’t use the getCharacter endpoint in this post. We will use it when we create the Word view component.)
The commit for this post is here.