The Basics of the MMAPI for Java Developers

This chapter is from the book 

Support multimedia playback and capturing in J2ME smart clients.

The successes of portable MP3 players and camera phones have proven the value of multimedia in mobile applications. Mobile users not only play back media contents on the go, but also share daily experiences via audio and video capturing. The J2ME Mobile Media API (MMAPI) enables multimedia applications on Java devices. It is supported on all Developer Platform 2.0 devices and many Developer Platform 1.0 devices. In this chapter, we cover the following topics:

• Introduction to the MMAPI: covers the basics of the API and supported media formats.
• Simple audio playback: uses a MIDI player example to show how to play back simple audio content from a local file or over the network. We discuss various player controls available on Series 40 devices.
• Advanced media playback: goes beyond MIDI and tones to show how to play back media files with more complex audio and video formats.
• Media capturing: uses a multimedia blog example to illustrate the use of the audio and video capturing API and how to share the captured data over the network.

The second half of this chapter ("Advanced Playback" and "Media Capturing") uses examples running on today's Series 60 devices rather than Series 40 devices. That helps the Series 40 developers to keep up with coming advances, since the MMAPI support is fast evolving with each new device release. It also helps developers to scale existing Series 40 applications up to higher end Series 60 devices. This chapter shows you the capabilities and programming techniques of the MMAPI on Nokia Series 40 and Series 60 Developer Platform devices.

Introduction to the MMAPI

The MMAPI is designed to support multiple media content types and data capturing mechanisms. It bears a lot of resemblance to the Generic Connection Framework (GCF) discussed in Chapter 7, "Data Connectivity." A subset of the MMAPI for simple audio playback is included in the MIDP 2.0 specification. All Nokia MIDP 2.0 devices, however, also implement the full MMAPI v1.1 (JSR 135) specification. The MMAPI features a generic factory class that instantiates media player objects from URI locator strings, InputStream objects, or DataSource objects. The device MMAPI implementation provides the concrete player classes for supported media formats. A player exposes some of its application-specific features via the Control interface. For example, a tone player has ToneControl, and a video capture player has VideoControl and RecordControl. We can interact with a player via its controls. Figure 9-1 shows the overall architecture of the MMAPI.

The Manager Class

The Manager class is the static factory class in the MMAPI. The createPlayer() method is the factory method used to create Player instances.

Create Player from URI Locators

The most versatile version of the createPlayer() method takes in a URI locator string to specify the network location of the media file, or the data capturing mode, or the in-memory empty device type.

static Player createPlayer (String locator)

In the MMAPI, three types of URI locator strings are supported.

• For media playback, the URI could point to a media file available on a remote HTTP server. The server must return the correct MIME type in the HTTP header for the createPlayer() method to determine which player instance to instantiate. For example, the URI locator string http://host/sample.mid is typically associated with the audio/midi MIME type on HTTP servers, and it would result in an audio MIDI file player instance. The supported MIME types in Series 40 and Series 60 Developer Platform 2.0 devices are listed in Table 9-1.

  Table 9-1. MIME Types in the Nokia MMAPI Implementation (varies among devices)

  MIME Types	Description
  audio/x-tone-seq	tone sequence
  audio/wav	wav audio format, but player cannot be created from InputStream using this MIME type
  audio/x-wav	wav audio format
  audio/au	au audio format
  audio/x-au	au audio format, but player cannot be created from InputStream using this MIME type
  audio/basic	raw audi format
  audio/amr	amr audio format
  audio/amr-wb	amr wb audio format
  audio/midi	midi audio format
  audio/sp-midi	extended midi format
  video/mp4	Mpeg4 video format
  video/mpeg4	mpeg4 video format, but player cannot be created from InputStream using this MIME type
  video/3gpp	3gpp video format
  application/vnd.rn-realmedia	real media video format

• For media capture, the URL string takes the special format capture://audio for audio capture or capture://video for still-image capture on a camera phone. The video mode displays video from the camera's viewfinder until you instruct the program to take a snapshot. Media capture is not supported in current Series 40 devices but is available on Series 60 devices and will be available on future Series 40 devices.
• For MIDI and tone sequence players, we can instantiate empty players in memory and then use MIDIControl and ToneControl objects to set content dynamically. The URI locator strings for such empty players are device://midi, which corresponds to the static value Manager.MIDI_DEVICE_LOCATOR, and device://tone, which corresponds to Manager.TONE_DEVICE_LOCATOR.

Create Player from Data Stream and MIME Type

The URI locator–based approach is simple and powerful. However, this approach relies on the server to provide the correct MIME types. If the server is configured incorrectly, or if the network protocol does not support MIME metadata (e.g., Unix socket), we cannor create the correct player. More importantly, not all media data is available over the network. We might need to play back locally stored media data, such as files bundled in the JAR file or data arrays stored in the RMS store. To address the above issues, we need a way to assign any MIME type programmatically to any data input stream available on the device. The second version of the createPlayer() method allows us to do just that.

static Player createPlayer (InputStream is, String type)

Refer to Table 9-1 for the list of supported MIME types. Please note that each individual device supports only a subset of those types. For example, most current Series 40 devices support only the audio/x-tone-seq, audio/midi, and audio/sp-midi MIME types. More detailed information is available in the Mobile Media API Technical Note published on the Forum Nokia Web site. If a null value is passed as the MIME type parameter, the implementation should try to figure out the media type based on its content. If the actual data in the InputStream is not encoded in the specified MIME format, or if the implementation cannot determine the media format when a null type parameter is passed, a MediaException may be thrown at runtime. We cannot instantiate data capture players with this method.

Create Player from DataSource

The third version of the createPlayer() method takes in a DataSource object to create a player. The DataSource class defines several abstract life cycle methods, which allow the users to specify how to connect to a custom data source and start or stop data transfer. A DataSource instance contains one or multiple SourceStream objects, which manage the actual media data. The SourceStream is different from the InputStream in the following aspects.

• SourceStream supports an API for random seeking that is required by some custom media data protocols.
• SourceStream defines abstract methods to support the concept of transfer size that is more suited for frame-delimited data (e.g., video).

Both the DataSource and SourceStream classes are abstract. They provide a framework for users to extend the MMAPI to support custom media data protocols. They are rarely used. We do not cover them in detail in this chapter.

Other Manager Methods

In addition to creating new player instances, we can use the Manager class to query the supported media types and protocols in this MMAPI implementation. Check out the MediaPlayer example later in this chapter to see those methods in action.

// Returns the supported media types for a given protocol
static String [] getSupportedContentTypes (String protocol)
// Returns the supported protocols for a given media type
static String [] getSupportedProtocols (String type)

The Manager class can also play tones directly to the device's speaker. This call does not block.

static void playTone (int tone, int duration, int vol)

The duration argument is the duration of the sound in milliseconds; the vol argument is the playback volume from 0 to 100; the tone argument takes a value from 0 to 127. The relationship between the tone value and the resultant frequency is as follows:

tone = 12 * log2 (freq/220) + 57

For example, the MIDI tone value 69 corresponds to the frequency of 440 Hz, which is the musical note A4. Table 9-2 shows musical notes and their corresponding frequencies and MIDI tone values. To get notes that are one octave higher or lower, we can add or subtract 12 from the MIDI tone values. Or, we can double or half the frequencies.

Table 9-2. Musical Notes and Their Corresponding Frequencies and MIDI Tone Values

Note	Frequency (Hz)	MIDI Tone
A4	440.00	69
A#	466.16	70
B	493.88	71
C	523.25	72
C#	554.36	73
D	587.33	74
D#	622.25	75
E	659.25	76
F	698.45	77
F#	739.98	78
G	783.99	79
G#	830.60	80

System Properties

The MMAPI specification leaves a lot of flexibility to implementers. For example, Nokia can decide what features and encoding types to support on each MMAPI-compatible device. As we have discussed, the Manager.getSupportedContentTypes() and Manager.getSupportedProtocols() static methods help us to determine the capabilities of the MMAPI implementation. In addition, the MMAPI implementation provides a number of system properties that can be retrieved via the System.getProperty() static method. Those properties give us information about the MMAPI implementation. Table 9-3 describes those properties and their values on Nokia 6230 (Series 40) and Nokia 6600 (Series 60) devices. The MediaPlayer example later in this chapter provides a utility to query MMAPI implementation capabilities based on the above methods and system properties.

Table 9-3. System Properties in the Nokia MMAPI Implementation

System Properties	Description	Nokia 6230	Nokia 6600
supports.mixing	Query for whether audio mixing is supported.	false	false
supports.audio.capture	Query for whether audio capture is supported.	false	true
supports.video.capture	Query for whether video capture is supported.	false	true
supports.recording	Query for whether recording is supported.	false	true
audio.encodngs	The string returned specifies the supported capture audio formats.		encoding = pcm encoding = ulaw encoding = alaw encoding = wav
video.encodings	The string returned specifies the supported capture video formats (video recording).
video.snapshot.encodings	Supported video snapshot formats for the VideoControl.getSnapshot() method.		encoding = jpeg encoding = bmp encoding = png
microedition.media.version	Returns 1.1 for an implementation that is compliant with MMAPI v1.1.	1.1	1.0
streamable.contents	Returns formats that can be streamed. No streaming format is supported at this time.

Player

The MMAPI specification declares the Player interface, which specifies the common behavior of all Player implementation classes provided by MMAPI implementers (e.g., Nokia) to handle different media types. The most important attribute of a Player is its life cycle states.

Player Life Cycle

A Player object can have the following states. Figure 9-2 illustrates the state transitions.

• CLOSED: The player has released most of its resources, and it can never be used again. We can change the player from any other state to the closed state by calling the Player.close() method.
• UNREALIZED: The player object has just been instantiated in the heap memory. It has not allocated any resources.
• REALIZED: If the Player.realize() method is called in a unrealized state, the player acquires required media resources and moves itself to the realized state. For example, if the player plays a remote media file over the HTTP network, the entire file is downloaded during the realizing process.
• PREFETCHED: If the Player.prefetch() method is called, the player performs a number of potentially time-consuming startup tasks and moves itself to the prefetched state. For example, during the prefetching process, the player acquires controls over the camera, the speaker, or other exclusive resources. The prefetching could fail if another program is already using some of those recourses. If the failure happens, we can call prefetch() later again on the same player. Once the player is prefetched, it can be started without further delay. Theoretically, we should move a prefetched player back to realized state by calling Player.deallocate(), but this method has not been implemented in Nokia devices.
• STARTED: By calling the Player.start() method, we can start the player, which starts the media playback or starts the capture player. Once the player is started, we can also call the Player.stop() method to stop it and return it to the prefetched state. A stopped player can be started again, and it will resume playing from the point at which it was stopped.

PlayerListener

We can listen for the player's events by registering PlayerListener objects to a player instance. The PlayerListener interface declares only one method, playerUpdate(), which is invoked every time the registered player receives an event. The caller Player object passes the event and any application-specific data. Developers decide how to respond to the event by implementing this method.

void playerUpdate (Player player,
                     String event, Object data)

The event strings are defined as static variables in the PlayerListener interface. Most of them are self-explanatory: BUFFERING_STARTED, BUFFERING_STOPPED, CLOSED, DEVICE_AVAILABLE, DEVICE_UNAVAILABLE, DURATION_UPDATED, END_OF_MEDIA, ERROR, RECORD_ERROR, RECORD_STARTED, RECORD_STOPPED, SIZE_CHANGED, STARTED, STOPPED, STOPPED_AT_TIME, and VOLUME_CHANGED. Following are a couple of points to notice:

• Player state changes have their corresponding events, such as CLOSED, STARTED, and STOPPED. The player life cycle method always returns immediately, and we can process state changes asynchronously.
• A player could be stopped under several conditions. The END_OF_MEDIA event occurs when the entire media content is played back. The STOPPED_AT_TIME event occurs when the player is stopped at a preset time in a StopTimeControl (discussed later). The STOPPED event occurs only when the player's stop() method is invoked.
• The DEVICE_UNAVAILABLE event occurs when there is an incoming call. The DEVICE_AVAILABLE event occurs when the call is ended.

The Player class provides methods to register and remove the PlayerListener objects.

void addPlayerListener (PlayerListener listener)
void removePlayerListener (PlayerListener listener)

Other Methods in the Player Interface

The Player class supports methods to query the status of the current media file.

String getContentType ()
long getDuration ()
long getMediaTime ()
int getState ()
TimeBase getTimeBase ()

The following methods set how many times the player will loop and play the content, the media time of the current play position, and a new TimeBase to synchronize this player with another one. Please note that the current Series 40 devices only support the system time base.

void setLoopCount (int count)
long setMediaTime (long now)
void setTimeBase (TimeBase master)

Control

The Control interfaces in MMAPI allow developers to control aspects of media-specific players programmatically. The same class can implement multiple Control interfaces for maximum API flexibility. Since the Player interface inherits from the Controllable interface, every Player class implements the following methods, which return the Control objects supported by this particular Player class.

Control getControl (String type)
Control [] getControls ()

Control objects are identified by the type strings. For example, the following code retrieves a VolumeControl object from an audio player and then adjusts the volume level.

VolumeControl vc = player.getControl ("VolumeControl");
vc.setLevel (50);
player.start ();

The MMAPI defines many player controls. However, only a subset of them is supported on the current Nokia devices. Table 9-4 lists the controls supported by different types of players on Nokia devices. Now we have covered the basics of the MMAPI; in the next two sections, we give concrete examples to show how to use the API.

Table 9-4. Players and Controls on Nokia MMAPI Implementation

MIME Types	Series 40 Controls	Series 60 Controls
audio/x-tone-seq	ToneControl, TempoControl, RateControl, PitchControl, VolumeControl, StopTimeControl	VolumeControl, StopTimeControl, ToneControl
audio/wav, audio/au, audio/amr	n/a	VolumeControl, StopTimeControl, RecordControl
audio/x-wav, audio/basic, audio/x-au, audio/amr-wb	n/a	VolumeControl, StopTimeControl
audio/midi, audio/sp-midi	MIDIControl, TempoControl, RateControl, PitchControl, VolumeControl, StopTimeControl
video/mp4, video/mpeg4, video/3gpp, application/vnd_rn-realmedia	n/a	VolumeControl, StopTimeControl, VideoControl
Video Capture	n/a	VideoControl, StopTimeControl