Review the Optimization Log File

When you run the classifier set optimization task, IDOL Speech Server writes information about the optimization performance to a log file. To specify the log file, use the OutputLog parameter in the [lidoptimizer] section of the tasks configuration file (speechserver-tasks.cfg). To retrieve this log file, use the GetResults action, and set the Token action parameter to the token for the optimization task.

The log file shows how the language classifiers are performing on each iteration of the optimization. The <lidoptimize_iteration_record> summarizes the system performance across all languages for a specific iteration. For example:

<lidoptimize_iteration_record>
   <iteration>11</iteration>
   <learning_rate>0.923</learning_rate>
   <avg_score>0.951</avg_score>
   <min_score>0.847</min_score>
   <std_dev>0.0107</std_dev>
   <wt_nincreased>9</wt_nincreased>
   <wt_increased>ENUK,ENUS,ESES,ESLA,ARMSA,FRFR,DEDE,NLNL,ITIT</wt_increased>
   <wt_ndecreased>1</wt_ndecreased>
   <wt_decreased>PLPL</wt_decreased>
   <wt_avgchange>0.000939</wt_avgchange>
   <wt_avgchange_abs>0.000974</wt_avgchange_abs>
   <result_delta>0.000442</result_delta>
   <best_so_far>No</best_so_far>
</lidoptimize_iteration_record>

The record contains the following elements.

iteration The current iteration of the optimization process.
learning_rate The current learning rate of the algorithm. The learning rate decreases throughout the process.
avg_score The average correct score across all languages, with the current weighting.
min_score The score of the worst performing language classifier.
std_dev The standard deviation across the classifier scores.
wt_nincreased The number of languages whose weights were increased.
wt_increased The language pack codes for the languages whose weights were increased.
wt_ndecreased The number of languages whose weights were decreased.
wt_decreased The language pack codes for the languages whose weights were decreased.
wt_avgchange The average weight change in real terms.
wt_avgchange_abs The average weight change in absolute terms.
result_delta The delta in the result when compared to the previous iteration.
best_so_far Whether this iteration was the best iteration.

The aim of the optimization process is to optimize the values for avg_score, min_score, and std_dev.

The log file also contains <lidoptimize_lang_record> items. This item shows the performance of a specific language at a specified optimization iteration. For example:

<lidoptimize_lang_record>
   <iteration>11</iteration>
   <lang>HBIL</lang>
   <weight>1.031</weight>
   <score>0.947</score>
   <update_avgTrueClarity>1.105</update_avgTrueClarity>
   <update_minTrueClarity>1.040</update_minTrueClarity>
   <update_avgFalseClarity>1.110</update_avgFalseClarity>
   <update_minFalseClarity>1.041</update_minFalseClarity>
   <closest_imposter>ARMSA (33.3%)</closestImposter>
   <weight_pos1>0.934</weight_pos1>
   <weight_pos2>0.994</weight_pos2>
   <weight_neg1>1.133</weight_neg1>
   <weight_neg2>1.069</weight_neg2>
   <rel_change>0.001</rel_change>
   <rel_change_str>INCREASED</rel_change_str>k
   <next_weight>1.032</next_weight>
   <min_conf>1.011</min_conf>
   <low_conf>1.105</low_conf>
   <med_conf>1.157</med_conf>
   <high_conf>1.204</high_conf>
   <max_conf>1.318</max_conf>
   <false_conf>1.000</false_conf>
</lidoptimize_lang_record>

The record item shows the existing weight and the new weight for the language, as well as some information on intermediate values used to calculate it. The record also gives information on how the language performed on the last iteration, information about confusability and the closest imposter for the language, and details about the spread of confidence (from the best result to the worst).

iteration The current iteration of the optimization process.
lang The language that this record represents.
weight The weight for this language classifier at this iteration.
score The correct rate for this classifier at this iteration (that is, is the proportion of audio segments for this language that were correctly identified).
next_weight The new weight for this classifier, for the next iteration.
min_conf The minimum confidence seen for a true hit for this language.
low_conf A low confidence value seen for a true hit for this language.
medium_conf A medium confidence value seen for a true hit for this language.
high_conf A high confidence value seen for a true hit for this language.
max_conf The maximum confidence value seen for a true hit for this language.
false_conf The maximum confidence seen for a false hit with this classifier.

After all optimization iterations are complete, the <lidoptimize_result> log item reports the best iteration. For example:

<lidoptimize_result>
	<best_iteration>4</best_iteration>
</lidoptimize_result>

IDOL Speech Server writes the language classifier weights for the best iteration to the classifier list file. When you use this file to run language identification tasks, IDOL Speech Server balances the classifier scores based on the new, trained weights.


_HP_HTML5_bannerTitle.htm