After you prepare the classifier list, you can run a language identification task. However, before you do so, recommends that you optimize the language weights and score thresholds in the classifier list.
Optimization balances the language models. After training, some classifiers might be stronger than others owing to the properties of the training material and the languages themselves. The optimization process weights the language models so that weaker languages have increased recall, without compromising the precision for stronger language models.
Optimization also calculates score thresholds for open set language identification. In open set language identification, HPE IDOL Speech Server checks whether a particular identified language matches the audio with a score above the specified threshold. If the language score is lower than the threshold, it returns the language identification as unknown.
The optimization process requires several audio files, each from a known language. It analyzes sections throughout each file, and determines whether each section was identified correctly. If a particular language is frequently not identified, HPE IDOL Speech Server usually increases the weighting of the language classifier. Conversely, if HPE IDOL Speech Server frequently identifies a particular language more often than it occurs, it usually reduces the weighting of the language classifier.
To optimize a language set
Create a list that contains the name of each feature file and a label that specifies the language that the file represents. Use the following format:
LanguageName;LifFilename
You can specify multiple files for each language. Either type the file names in a comma-separated list, or add multiple instances of a language. For example:
ENUK;english1.lif ENUK;english2.lif ENUS;us-english1.lif,us-english2.lif ESES;spanish.lif FRFR;french.lif
You can also use data for unknown languages, to optimize open set language identification. In this case, use the tag unknown. For example:
Unknown;unknown1.lif
Unknown;unknown2.lif
For more information about HPE IDOL Speech Server's list manager, see Create and Manage Lists.
Run the optimization task. Send an AddTask
action to HPE IDOL Speech Server, and set the following parameters:
Type
|
The task name. Set to LangIdOptimize . |
DataList
|
The list that specifies the feature files. |
DataPath
|
The path to the directory that contains the feature files listed in the DataList . |
ClassList
|
The list that specifies the classifiers (see Combine Classifiers into a Language Identification Set). |
ClassPath
|
The path to the directory that contains the language classifiers. |
Out
|
The classifier list to write the optimized weights to. |
For example:
http://localhost:13000/action=AddTask&Type=LangIdOptimize&DataList=ListManager/OptList&DataPath=C:/LangID/Data&ClassList=ListManager/ClassList&ClassPath=C:/LangID/class&Out=ListManager/OptClassSet
This action uses port 13000
to instruct HPE IDOL Speech Server, which is located on the local machine, to test the feature files listed in the OptList
list against the classifiers specified in the ClassList
list and then generate an updated classifier list–OptClassSet
.
This action returns a token. You can use the token to:
The LangIdOptimize
action also generates a log file that contains information about the optimization process. See Review the Optimization Log File.
|