How to obtain timestamp information from your H264 stream (inside an Mpeg1 transport stream)

Some more information on parsing an H264 video stream to pass the individual NAL units to Silverlight's in-built decoder:

In my last blog post, I suggested incrementing the timestamp for each NAL picture frame. However, this technique isn't reliable since it relies on knowing the frame rate in advance and this isn't always available within an H264 stream. (H264 has no knowledge of frame rate as standard, although this info can be optionally included in Sequence Parameter Set units)

Therefore, I would suggest a better technique for obtaining timestamp information when decoding H264 video frames: Simply get your timestamps from the PES headers of the transport stream itself.

Apple's documentation relating to HTTP Live Streaming suggests that NAL picture frames (i.e. NAL units types 1-5) should be in 'unique PES packets'. This makes a lot of sense, since it means that the Transport Stream can then be used as a single reliable source of timestamps.

Use a decoding loop as follows when parsing an H264 byte stream: (assuming you already have code in place to demux the transport stream container)

• PES header encountered.
• Store the PTS/DTS from the PES header.
• Parse the PES stream for NAL units.
• Generate a MediaStreamSample correponding to each NAL unit encountered within the stream, timestamping it with the stored Dts (if available, or use Pts) values.
• If you encounter more than one NAL picture frame within the PES, it is an invalid stream. (you could try to guess the timestamp)
• PES header encountered; update your stored timestamp.
• Continue in this fashion until you reach the end of the stream