Firelight Technologies FMOD Studio API

HTML5 Specific Starter Guide

FMOD has been ported to HTML5 (JavaScript) for use in a web page or java app. FMOD.JS or FMODSTUDIO.JS can be included directly into a web page to play interactive audio within a browser.

FMOD Studio for HTML5 is fully functional but has some limitations due to JavaScript.

FMOD does not run in a 'thread' in HTML5 so has to execute from the game loop. This includes all loading and mixing/decoding/DSP. Check the performance section for more on this.

Technologies supported by HTML5, what to use

There are 3 ways to include FMOD in your project.

  1. A JS file included in a HTML file, using the older ASM.JS technology. Slower and uses more memory, but best for compatibility.
  2. A JS file included in a HTML file, using the newer WASM technology. Faster (approximately 30%) and uses half the memory of ASM.JS, but requires the web browser to support the WASM technology, and also a webserver that understands the .wasm 'application/wasm' mime type.
  3. A BC file included in an emscripten project, using C++. This is if you are using emscipten with your own C/C++ code to convert to JavaScript. BC is an intermediate format.

To include FMOD using JavaScript in a html file, use the following code in your HTML.

Low Level API Only

<script type="text/javascript" src="fmod.js"></script>

FMOD Studio API

<script type="text/javascript" src="fmodstudio.js"></script>

Which files do you include?

Below are the 3 categories of files you can include based on the 3 categories listed above (ASM.JS, WASM, BitCode).
You can either use the FMOD Low Level API only, or the FMOD Studio API.


ASM.JS - FMOD Low Level API

ASM.JS - FMOD Studio API


WASM - FMOD Low Level API

WASM - FMOD Studio API


Emscripten Bit Code Libraries - - FMOD Low Level API

Emscripten Bit Code Libraries - FMOD Studio API


Note. The FMOD Studio API .js (and .bc) files already contain the low level API, so you do NOT need to include the low level API files as well. Note2. The fmodstudio.js or .bc file links to the fmod_reduced version of the low level, so accessing things like .ogg/.mp3/convolution reverb (see reduced ) through System::getLowLevelSystem are not supported.

SDK Version

FMOD is compiled using the following tools.

Browser Compatibility

FMOD for HTML5 has been tested on the following browsers and versions with the following results.

Browser OS Version 5.1 / Surround Sound Result
Google Chrome Windows 55 Fail Successful
Google Chrome Windows 66 Fail Successful with user interaction workaround (See Essential Startup Information)
Mozilla Firefox Windows 50.1.0 Successful Successful
Microsoft Edge Windows 38.14393.0.0 Fail Successful
Microsoft Internet Explorer Windows 11.576.14393 Fail Fail
Opera Windows 42.0 Fail Successful
Safari Mac 10.0.2 Successful
Google Chrome Mac 55 Fail Successful
Google Chrome Mac 66 - Successful with user interaction workaround (See Essential Startup Information)
Google Chrome Android (Galaxy S6) 55 - Successful
Safari iOS - Successful with user interaction workaround (See Essential Startup Information)

Essential Startup Information

Start up code

To start with FMOD in a JavaScript environment you must initialize a global FMOD object, which you can then call FMOD functions with.

This involves declaring a variable (ie 'FMOD') then calling a constructor function on it ( 'ie FMODModule(FMOD); )

There are 2 optional functions which can be used to set up 'pre-runtime' information like file loading, and

var FMOD = {};                          // FMOD global object which must be declared to enable 'main' and 'preRun' and then call the constructor function.
FMOD['preRun'] = prerun;                // Will be called because FMOD runs, but after the Emscripten runtime has initialized
FMOD['onRuntimeInitialized'] = main;    // Called when the Emscripten runtime has initialized
FMOD['TOTAL_MEMORY'] = 64*1024*1024;    // (ASM.JS ONLY) FMOD Heap defaults to 16mb which is enough for this demo, but set it differently here for demonstration (64mb)
FMODModule(FMOD);                       // Calling the constructor function with our object

Safari and Chrome Browser user interaction requirement (use for all browsers)

Safari (on iOS) and Chrome Browser (version 66 and above) both have a user interaction requirement, or audio will not be audible. This was implemented to stop unscrupulous websites from auto playing audio in things like advertisments.

The workaround is that a web app must start an audio context from a touch event or a mouse click event, meaning there was user interaction. FMOD's mechanism for doing this currently is to call System::mixerSuspend followed by System::mixerResume. This 'reinitializes' the webaudio driver and because it is in the context of a user interaction, the audio will start successfully.

It is up to the developer to determine where the click/touch must be at the start of the application, this will have to be part of the application design. No audio is possible without it, so traditional splash screens or introductions are not possible with audio.

FMOD examples all have code that can be copied into a developer's application. Here is the relevant JavaScript code.

// Set up iOS/Chrome workaround.  Webaudio is not allowed to start unless screen is touched or button is clicked.
function resumeAudio() 
{
    if (!gAudioResumed)
    {
        console.log("Resetting audio driver based on user input.");

        result = gSystem.mixerSuspend();
        CHECK_RESULT(result);
        result = gSystem.mixerResume();
        CHECK_RESULT(result);

        gAudioResumed = true;
    }
}

var iOS = /iPad|iPhone|iPod/.test(navigator.userAgent) && !window.MSStream;
if (iOS)
{
    window.addEventListener('touchend', resumeAudio, false);
}
else
{
    document.addEventListener('click', resumeAudio);
}

Running examples on your local machine

A web browser is typically not allowed to access your harddisk directly, it has to do it through a web service. This means FMOD examples won't run without a local web server if running the examples locally. If you are used to development in a web browser this should be obvious by now.

An example, install Apache for your operating system, or for windows, a simple web server for chrome link

Reduced Feature build (fmod_reduced.js and fmodstudio.js) feature removal.

Overview

The fmod_reduced library is a smaller subset of features. The following is a list of what is removed.

fmodstudio.js links to fmod_reduced.js by default. To get the full feature set use fmodstudioL.js instead.

Codec Support

The following are removed in favour of .FSB only. .FSB support includes vorbis, fadpcm and pcm compression formats.

DSP Effect support

The following are removed due to relative expensiveness of the effect and usage rates being below other types of effects.

Speaker mode support

Feature support

The following features have been removed

API - JavaScript port of a C/C++ API. Differences.

Setting and getting.

This is an important section if coming from knowledge of a C/C++ background of the FMOD API.

Javascript parameters are passed by value, therefore when you pass one to a function, it makes the concept of a 'getter' function difficult. The variable's value cannot be changed by the function from the caller's perspective, but it can add a new member to the variable, which is the mechanism FMOD always uses when 'getting' data from a function.

In C the variable can be altered as it would be passed by reference (using pointers). In FMOD for JavaScript, the variable you pass in gets a new member called val which contains the new data.
i.e.

var outval;    // generic variable to reuse and be passed to FMOD functions.
var name;      // to store name of sound.

sound.getName(name);
name = outval.val;  // 'val' contains the data.  Pass it to the variable we want to keep.

console.log(name);

All FMOD functions that produce data in a variable after calling a function return data this way.

Constants that started with FMOD_ in C/C++

In FMOD for JavaScript the enumerated values are members of the object declared at the top of the file.
As this would normally be a variable called 'FMOD' it would be redundant to keep the FMOD_ as part of the constant name, so it is removed to avoid duplication. For example, instead of

FMOD.FMOD_OK

in FMOD for HTML5 it becomes:

FMOD.OK

similarly

FMOD_INIT_NORMAL
FMOD_DEFAULT
FMOD_ERR_FILE_NOTFOUND
FMOD_GUID
FMOD_3D_ATTRIBUTES
etc

becomes

FMOD.INIT_NORMAL
FMOD.DEFAULT
FMOD.ERR_FILE_NOTFOUND
FMOD.GUID()
etc

Not that for attributes that start with a number after the FMOD namespace, it has to start with an underscore. For example.

FMOD._3D
FMOD._2D
FMOD._3D_ATTRIBUTES()


Using structures

In the above example you may notice that () is used to 'construct' a javascript object which represents a C structure in the FMOD API. When using a structure to pass information to FMOD, a helper/constuctor function must be called to create the structure before using it/filling in its members, so that FMOD can understand what is being passed to it. If these constructor functions are not used, the function it is being passed to will probably result in a 'binding' error (in the browser debug/log console/window).

var guid = FMOD.GUID();
var info = FMOD.STUDIO_BANK_INFO();


API differences

Some API constructs are redundant for JavaScript, so are removed from the API.

Examples are 'cbsize' inside a struct. The JavaScript version of a struct already knows its own size, so it is removed/redundant here.

Structures like FMOD.CREATESOUNDEXINFO, FMOD.ADVANCEDSETTINGS and FMOD.STUDIO_ADVANCEDSETTINGS have these members, it can just be left out.

File Access

FMOD lets you load data from the host in a few different ways, depending on your setup.

Direct from host, via FMOD's filesystem

FMOD has a mechanism to mount/pre-load files in the 'prerun()' function, that is described above in the "Essential Startup Information" section of this document. Call FMOD.FS_createPreloadedFile to register your files so that FMOD can use filenames in file related functions. For example the playsound example

// Will be called because FMOD runs, but after the Emscripten runtime has initialized
// Call FMOD file preloading functions here to mount local files.  Otherwise load custom data from memory or use own file system. 
function prerun() 
{
    var fileUrl = "/public/js/";
    var fileName;
    var folderName = "/";
    var canRead = true;
    var canWrite = false;

    fileName = [
        "dog.wav",
        "lion.wav",
        "wave.mp3" 
    ];

    for (var count = 0; count < fileName.length; count++)
    {
        FMOD.FS_createPreloadedFile(folderName, fileName[count], fileUrl + fileName[count], canRead, canWrite);
    }    
}

Then later in your app you can simply reference a file by path/filename.

result = gSystem.createSound("/lion.wav", FMOD.LOOP_OFF, null, outval);
CHECK_RESULT(result);


Via memory

If you have raw data in memory that you want to pass to FMOD , you can. Warning though, javascript passes data by value, so the memory usage will temporarily double for the sound data when it is passed to fmod. The load_from_memory has an example of this

var chars  = new Uint8Array(e.target.result);
var outval = {};
var result;
var exinfo = FMOD.CREATESOUNDEXINFO();
exinfo.length = chars.length;

result = gSystem.createStream(chars.buffer, FMOD.LOOP_OFF | FMOD.OPENMEMORY, exinfo, outval);
CHECK_RESULT(result);


Via callbacks

If you have a file system you can use file callbacks to load data into FMOD. See System::setFileSystem, or Studio::System::loadBankCustom In load_bank example, there is a demonstration of this

var info = new FMOD.STUDIO_BANK_INFO();

info.opencallback = customFileOpen;
info.closecallback = customFileClose;
info.readcallback = customFileRead;
info.seekcallback = customFileSeek;
info.userdata = filename;

result = gSystem.loadBankCustom(info, FMOD.STUDIO_LOAD_BANK_NONBLOCKING, outval);
CHECK_RESULT(result);

In this case the filename is passed as userdata to let the callback get information for what file to load. Here is the customFileOpen callback getting the filename and opening a file handle. The file is opened in this case with a special FMOD provided file API. Replace this with your own API.

function customFileOpen(name, filesize, handle, userdata)
{
    var filesize_outval = {};
    var handle_outval = {}

    // We pass the filename into our callbacks via userdata in the custom info struct
    var filename = userdata;

    var result = FMOD.file_open(gSystemLowLevel, filename, filesize_outval, handle_outval)
    if (result == FMOD.OK)
    {
        filesize.val = filesize_outval.val;
        handle.val = handle_outval.val;
    }

    return result;
}

To read data in the callback, into the buffer that FMOD has provided, FMOD has used a special technique where it passes the memory address, instead of a javascript buffer. This means to write data to FMOD's buffer you must use FMOD.setValue using the buffer address, writing the data in a loop, a byte at a time. This could be optimized somewhat by using i32 to write 4 bytes at a time.

function customFileRead(handle, buffer, sizebytes, bytesread, userdata)
{
    var bytesread_outval = {};
    var buffer_outval = {};

    // Read from the file into a new buffer.  This part can be swapped for your own file system.
    var result = FMOD.file_read(handle, buffer_outval, sizebytes, bytesread_outval)   // read produces a new array with data.
    if (result == FMOD.OK)
    {
        bytesread.val = bytesread_outval.val;
    }

    // Copy the new buffer contents into the buffer that is passed into the callback.  'buffer' is a memory address, so we can only write to it with FMOD.setValue
    for (count = 0; count < bytesread.val; count++)
    {
        FMOD.setValue(buffer + count, buffer_outval.val[count], 'i8');      // See https://kripken.github.io/emscripten-site/docs/api_reference/preamble.js.html#accessing-memory for docs on setValue.
    }

    return result;
}

JavaScript specific functions for FMOD

Helper functions

FMOD comes with some functions to aid with reading file data and writing to memory buffers. Here is the list of functions that are provided.

System

FMOD['preRun'] = callback                // Variable to assign a function callback to.  Will be called before FMOD runs, but after the Emscripten runtime has initialized
FMOD['onRuntimeInitialized'] = callback  // Assign a function to this member of FMOD, pre Called when the Emscripten runtime has initialized **
FMODModule(fmodobject)                   // Constructor function to set up FMOD object before use.  fmodobject = the main FMOD object you will use throughout the application lifetime.

FMOD.FS_createPreloadedFile              // Mounts a local file so that FMOD can recognize it when calling a function that uses a filename (ie loadBank/createSound)
FMOD.System_Create                       // Only exported in FMOD.JS or FMODL.JS (Only use this functio if using pure Low Level API).  See docs for FMOD::System_Create
FMOD.Studio_System_Create                // Only exported in FMODSTUDIO.JS or FMODSTUDIOL.JS (Only use this function if using Studio API). See docs for FMOD::Studio::System_Create
FMOD.ReadFile                            // Read the entire contents of a file into a memory variable, as preloaded by FMOD.FS_createPreloadedFile.  Call FMOD.Memory_Free on the variable after using it.
FMOD.Memory_Free                         // Free memory allocated by FMOD internally in FMOD.ReadFile

File

FMOD.file_open
FMOD.file_close
FMOD.file_read
FMOD.file_seek

Misc

FMOD.setValue                           // See https://kripken.github.io/emscripten-site/docs/api_reference/preamble.js.html#accessing-memory for docs on setValue.

Linking FMOD for HTML5 in a C program that is to be converted via emscripten

You should be able to port existing C code with no special HTML5 related functionality. Link the fmod.bc file or the fmodstudio.bc file (not both!) into your application to get it to link. As mentioned previously FMOD_NONBLOCKING will not be needed, so your program might hang if System::update is not processed during any logic that waits on a getState function.

Performance and Memory

CPU Overhead

By default FMOD mixes at 48000hz. If the browser is not running at that rate, it will introduce a resampler to convert the rate, which consumes CPU time and adds a DSP block worth of latency. This can be solved by querying the hardware's rate before System::init, and calling System::setSoftwareFormat to the same rate as the output. Here is an example (from the PlaySound example)

var outval = {};
result = gSystem.getDriverInfo(0, null, null, outval, null, null);
CHECK_RESULT(result);
result = gSystem.setSoftwareFormat(outval.val, FMOD.SPEAKERMODE_DEFAULT, 0)
CHECK_RESULT(result);

Audio Stability (Stuttering)

Some devices cannot handle the default buffer size of 4 blocks of 1024 samples (4096 samples) without stuttering, so to avoid this set the buffer size in your application to 2 blocks of 2048. Here is an example

result = gSystem.setDSPBufferSize(2048, 2);
CHECK_RESULT(result);

Threads

As stated in the introduction, FMOD for HTML5 does not have access to any threads so any loading/mixing/decoding/streaming/dsp has to be run in a blocking mode, from the main application loop.

Keep this in mind when loading sounds, or implementing DSP that may take a large time slice. If the application pre-loads sounds, and has a fast enough framerate that the FMOD mixing can execute in the same frame (with time left over) then there should be no audible glitching or frame rate glitching.

'Heap' Memory

FMOD defaults to 16mb of memory to load sounds with and create FMOD objects with. Use the following global before calling the main FMOD constructor object, to control how much memory to use.

FMOD['TOTAL_MEMORY'] = 64*1024*1024;    // (ASM.JS ONLY) FMOD Heap defaults to 16mb which is enough for this demo, but set it differently here for demonstration (64mb)

For WASM support, FMOD uses ALLOW_MEMORY_GROWTH, which is more dynamic, so TOTAL_MEMORY is ignored.

Limitations

Non-blocking loading

FMOD.NONBLOCKING and FMOD.EVENT_NONBLOCKING do not allow sound to load in the background, but it will process loading from the update loop instead.

Streaming is executed from the main thread. This may impact performance, so pre-loading sounds is prefered.

Streaming infers that the source is being loaded from 'disk' which is not usually done with HTML5, because data is usually pre-packaged with the javascript file, or streamed over http (from an external URL) so loading sounds into memory might make more sense here.

Known Issues

The following functions are not supported. Some have no place in JavaScript (ie loadPlugin) but others are incomplete at this point and can be added in a later point release.

The following callbacks remain unimplemented at this point, so they will not work.


Please contact support@fmod.com if you discover a problem.