Run Image Description

To run image description, follow these steps.

To run image description

Import the pre-trained model, by running the action ImportImageDescriptionModel. The model is not included in the Media Server package due to its size, but is available separately. For example:

curl http://localhost:14000/action=ImportImageDescriptionModel -F modelpath="C:\IDOL\ImageDescription_LLaVA-v1.6-Mistral-7B-HF.dat"

This action can take 2-3 minutes to complete. You can track its status by using the QueueInfo action to query the status of the ImportImageDescriptionModel queue. When the action has completed, you can run:
```
http://localhost:14000/action=listimagedescriptionmodels
```
Media Server returns the result, showing the imported model:
```
<autnresponse>
  <action>LISTIMAGEDESCRIPTIONMODELS</action>
  <response>SUCCESS</response>
  <responsedata>
    <model>
      <model>LLaVA-v1.6-Mistral-7B-HF</model>
    </model>
  </responsedata>
</autnresponse>
```

Create a session configuration that ingests the media and runs image description. There is an example configuration in the Media Server package at configurations/examples/ImageDescription. For example:

[Session]
Engine0 = Source
Engine1 = Describe
Engine2 = WriteText

[Source]
Type = image

[Describe]
Type = ImageDescription
Model = LLaVA-v1.6-Mistral-7B-HF
// Limit the length of text description
MaxOutputTokens = 100

[WriteText]
Type = xml
Input = Describe.Data
OutputPath = output/%source.filename.stem%.txt
// Extract just the description from output
XSLTemplate = toText.xsl

Run image description, by providing your session configuration and a media file to the process action.

curl http://localhost:14000/action=process -F configname=examples/ImageDescription/ImageDescription.cfg -F sourcedata=@image.jpg

When running on a CPU, image description can take several minutes to complete. You can track the status of your process action by using the QueueInfo action to query the status of the process action queue. When the action has completed, check the output directory for the results. You can repeat this step to run image description on as many images as necessary.