loliball/media3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Set hasMuxedTimestampZero to true only when its muxed

Browse Source 
The muxer might not have accepted the first sample, if it
is waiting for audio track.

This bug causes issue when

1. VideoSampleExporter gives first sample (timestamp = 0) to the muxer.
2. Muxer does not write it because its waiting for audio track.
3. The video pipleline has processed all the sample and they are ready
to be consumed.
4. VideoSampleExporter fetches the next available sample from encoder (which is still with timestamp = 0) but it changes its timestamp to last timestamp because VideoSampleExporter thinks it has muxed the sample at timestamp zero, but in reality it hasn't. This is because the flag `hasMuxedTimestampZero` is set when queueing the input, rather than actually muxing the input.

This scenario can happen when video is processed much faster than
the audio.

PiperOrigin-RevId: 675565603
This commit is contained in:
sheenachhabra 2024-09-17 07:50:04 -07:00 committed by Copybara-Service
parent e1c4ecf2d3
commit acb8e71c6e
1 changed files with 6 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
libraries/transformer/src/main/java/androidx/media3/transformer/VideoSampleExporter.java
View File

@ -73,6 +73,7 @@ import org.checkerframework.dataflow.qual.Pure;
*/
private volatile long finalFramePresentationTimeUs;
private long lastMuxerInputBufferTimestampUs;
private boolean hasMuxedTimestampZero;
public VideoSampleExporter(
@ -96,6 +97,7 @@ import org.checkerframework.dataflow.qual.Pure;
super(firstInputFormat, muxerWrapper);
this.initialTimestampOffsetUs = initialTimestampOffsetUs;
finalFramePresentationTimeUs = C.TIME_UNSET;
lastMuxerInputBufferTimestampUs = C.TIME_UNSET;
ColorInfo videoGraphInputColor = checkNotNull(firstInputFormat.colorInfo);
ColorInfo videoGraphOutputColor;
@ -191,17 +193,19 @@ import org.checkerframework.dataflow.qual.Pure;
&& finalFramePresentationTimeUs != C.TIME_UNSET
&& bufferInfo.size > 0) {
bufferInfo.presentationTimeUs = finalFramePresentationTimeUs;
} else {
hasMuxedTimestampZero = true;
}
}
encoderOutputBuffer.timeUs = bufferInfo.presentationTimeUs;
encoderOutputBuffer.setFlags(bufferInfo.flags);
lastMuxerInputBufferTimestampUs = bufferInfo.presentationTimeUs;
return encoderOutputBuffer;
}
@Override
protected void releaseMuxerInputBuffer() throws ExportException {
if (lastMuxerInputBufferTimestampUs == 0) {
hasMuxedTimestampZero = true;
}
encoderWrapper.releaseOutputBuffer(/* render= */ false);
videoGraph.onEncoderBufferReleased();
}