While AVAA Toolkit's core is made with Java, most features are actually implemented in JavaScript.

All these scripts are stored in the scripts folder and processed when necessary as the XML-to-HTML conversion advances.

The file name itself indicates the nature of the component, and its unique id.

This modularity allows anyone to create custom scripts to extend AVAA Toolkit and solve specific tasks, without the need for further core AVAA Toolkit development.

Scripts of common utility could then be shared by their authors and benefit the community at large.

Views

// file view.my-custom-view.js
view['my-custom-view'] = {
    list:list=>{
        let html='';
        for(a of list){
            html+=`<div class="annotation">${a.tier.id}: ${a.value}</div>`;
        }
        return html;
    },
    css:()=>{
        return `
            #view div {
                    color:green;
                    margin:10px;
                    border:2px dashed gray;
            }
        `
    }
}
                

Operations

/**
    @novalue
    A simple operation to reverse an input array
**/
function(value, data){
    if(!Array.isArray(data)){
        log.error(`input must be an array`);
        return data;
    }
    return data.reverse();
}
                

Processors

proc['my-custom-processor'] = {
    pipe:pipe=>{
        for(let f of pipe.corpus){
            let step = log.addStep(f);
            for(let mf of f.media){
                step.file(mf, "Found media file:");
            }
        }
    }
}
                

Mods

                    TODO
                

Each script file can begin with doc comments /** ... **/ containing meta definitions.

A definition is a line starting with a keyword prefixed by the @ character.

The following definitions are common to all script files:

• @author Script file author
• @license A license notice
• @log A log entry, usually to log major changes
• @todo A todo note
• @sample Sample usage for the doc
• @image Sample image (taken from dependencies/images folder)
• @experimental Indicates the component is a work in progress
• @hidden Whether to hide this component from the user (providers registering an extension use it to hide themselves from the providers' list)

The following definitions make sense to some components:

• @input Allowed input type
• @output Expected output type
• @attr Custom attribute
• @param Custom parameter
• @dependency Custom JS file to load from the dependencies folder before loading the script file itself
• @novalue Indicates the operation does not need an associated value

If a line does not start with the @ character, it will be considered part of the component's description.

When a component is executed, some helper objects are available in its scripting scope.

The following objects are available in all the components:

• log A logger to raise feedback to the user
• api The API to access core functions and state
• bag An object persistent across full conversion of the document
• variables An object with current user-defined global variables

The following objects are available depending on the component:

• data Current data being processed
• dataJSON Current data as a JSON string, if possible
• annotations Current selection (List) of annotations being processed, if available
• attr An object with all the defined attributes and their values
• params An array of parameters Element
• pipe Currently executing Pipeline

The ScriptAPI exposes an exec(args:Array) method to prepare a process execution.

let step = log.addStep({name:`Getting ffmpeg version`});
let proc = api.exec(['ffmpeg','-version'])
proc.onError(msg=>{
    step.error(`A process error occurred: ${msg}`)
})
proc.onOutput(msg=>{
    step.info(msg)
})

// monitors execution and allows killing of the process from editor
step.monitor(proc);

// run the process and get its exit status
let exitStatus = proc.run();
let benched = Math.ceil(proc.duration/1000)+1;
if(!exitStatus){
    step.info(`Completed in ${benched} seconds`);
    step.ok();
}else{
    step.error(`Process exited with error code ${exitStatus} after ${benched} seconds`);
}
    

It's possible to execute ffmpeg with the Generic Process approach, but it is cumbersome to read progress feedback and other results.

To this end, AVAA Toolkit provides a wrapper script that must be included in the component via the @dependency proc-ffmpeg-wrapper.js meta definition.

Then calling ffmpeg with

let step = log.addStep({name:`Custom Video Effect`});
let ffargs = [
    '-i', inputFile.getAbsolutePath(),
    '-filter_complex', `"gblur=sigma=42:steps=6;format=yuv420p"`,
    outputFile.getAbsolutePath()
]
step.info(`Applying Gaussian Blur...`)
step.work()
let result = ffmpegWrapper(step, ffargs)
if(!result) throw `Failed applying the video filter!`
            

The wrapper has additional options not yet covered in this guide.

AVAA Toolkit also provides a wrapper and a protocol to simplify calling Python programs and reading their output.

Because it is not possible to understand the output of all potential Python programs, authors wishing to interface a Python program with AVAA Toolkit have to prefix its output according to the following rules:

• #E: to indicate an error, like

  print('#E:missing input')

• #W: to send a warning, like

  print('#W:file will be overwritten')

• #I: to log information, like

  print('#I:Temperature =', temperature)

• #F: to link a file, like

  print('#F:Input =', args.input)

• #D: for download progress, like

  print('#D:20%')

• #P: for process progress, like

  print('#P:6/50/Encoding')

• #R: for the result, like

  print('#R:'+json.dumps(result))

Now to implement the processor that will call the Python program, it is necessary to include these two dependencies:

• @dependency proc-exec-output-wrapper.js This dependency provides the wrapper to interpret the output of the program.
  This output wrapper is not specifically for Python programs, it can be used for any program that wants to interface with AVAA Toolkit.
@dependency proc-python-wrapper.js This dependency provides helper functions to check the availability of Python itself, and installed Python modules via pip.

Let's see how this all fit with some code taken from the speech-to-text-whisper processor:

let req = log.addStep({name:'Requirements'})
let pythonVersion = pythonCheck(req);
if(!pythonVersion) return;
if(!pipCheck("torch",req)) return;
if(!pipCheck("openai-whisper",req)) return;

// checks that the avaa-whisper.py script is working
// the file avaa-whisper.py is in the dependencies folder
// it's a python script made to interface Whisper with AVAA
let whisperWrapper = pythonWrapper("avaa-whisper",req);

let step = log.addStep('Transcription');
let xargs = ['python','-u',whisperWrapper.getAbsolutePath(),
    '-i',inputFile.getAbsolutePath(),
    '-m',whisperModel,
    '-t',whisperTemp,
    '-l',whisperLang
];
let proc = api.exec(xargs);
step.monitor(proc);
proc.onError(s=>{
    step.error(`Process Error: ${s}`)
});
// here we use the output wrapper
proc.onOutput(execOutputWrapper(step,result=>{
    step.info(`Got results from transcription:`);
    let a = JSON.parse(result);
    // ...
}));
let exitStatus = proc.run();