18347 Commits

Author SHA1 Message Date
huangdarwin
518fe3771a HDR: Fix ForceInterpretHdrVideoAsSdrTest, and expand to all APIs.
Previously, this was limited to API 29. Expand this to all API versions.

Also, update the test to:
(1) skip based on SDR format input instead of HDR format input
(2) Check the exception message in order to disambiguate between the decoder tone
mapping error, and general video format support error.

PiperOrigin-RevId: 511002218
2023-02-27 18:12:06 +00:00
samrobinson
0ce1fcc4ad Release the created placeholder surface in DecodeOneFrameUtil.
PiperOrigin-RevId: 511000498
2023-02-27 18:11:18 +00:00
sheenachhabra
a33b0d64c1 Move SampleTableBoxesTest.java into third_party/.../muxer/...
Following changes are included:
1. Move test file into third_party/.../muxer/...
2. Rewrite tests for stts box and stsz box to use dump file infra instead of Mp4Slicer.
3. Remove existing test cases related to stts box which are kind of duplicate as it tries to cover the scenarios which are already covered by "DurationsVuForStts" related test cases.

PiperOrigin-RevId: 510977119
2023-02-27 18:10:23 +00:00
kimvde
ac12b96f11 Fix stuck muxer
Unstuck the muxer if the next timestamp in the track with the minimum
timestamp is larger than this minimum timestamp plus
MAX_TRACK_WRITE_AHEAD_US.

PiperOrigin-RevId: 510977088
2023-02-27 18:09:20 +00:00
huangdarwin
e282c0ad67 Effect: Add FloatRange to public GlEffect impl interfaces.
Using these annotations/checkers should hopefully make it marginally harder to use the wrong input values in the API.

PiperOrigin-RevId: 510966941
2023-02-27 18:08:28 +00:00
tonihei
34b9824201 Do not specify export flags for protected system broadcasts.
Protected system broadcasts should not specify the export flag.
Marking them as NOT_EXPORTED breaks sticky broadcasts in some
cases.

Issue: google/ExoPlayer#10970

#minor-release

PiperOrigin-RevId: 512020154
2023-02-27 18:08:18 +00:00
christosts
ab7e84fb34 Skip rendering multiple frames on the same vsync
When rendering frames at a rate higher than the screen refresh rate,
e.g. playing at 8x, the player is releasing multiple frames at the same
release time (nanos) which are then dropped by the platform. The output
buffers are available later and as a result MediaCodec cannot keep up
decoding fast enough.

This change skips releasing multiple video frames on the same vsync
period and proactivelly drops the frame. The frame is counted as skipped
rather than dropped to differentiate with frames dropped due to slow
decoding.

PiperOrigin-RevId: 510964976
2023-02-27 18:07:13 +00:00
tonihei
9eccf09165 Reduce number of calls to AudioTrack.getPlaybackHeadPosition
This call may cause performance overhead in some situations,
for example if the AudioTrack needs to query an offload DSP
for the current position. We don't need to check this multiple
times per doSomeWork iteration as the value is unlikely to
change in any meaningful way.

PiperOrigin-RevId: 510957116
2023-02-27 18:06:20 +00:00
huangdarwin
d1df8e21d2 HDR: Remove comment explaining force sdr behavior with assetloaders.
Before, if the upstream AssetLoader provides HDR to the VideoSamplePipeline when
HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR is requested, the
VideoSamplePipeline would attempt to tell the AssetLoader to output SDR, which
could be accomplished via MediaCodec tone-mapping in the AssetLoader.

However, this makes an assumption of the AssetLoader implementation, and
AssetLoaders may not all implement support for decoder tone-mapping. Remove javadoc
attempting to explain how AssetLoaders (ex. custom ones) could behave.

PiperOrigin-RevId: 510956820
2023-02-27 18:05:16 +00:00
samrobinson
4a7326a286 Pass AudioFormat to ASP#computeNextEncoderInputBufferTimeUs.
This method uses sampleRate, channelCount and pcmEncoding, so passing
AudioFormat is easier.

This will lead into a future change that builds the
encoderInputAudioFormat from encoder.getConfigurationFormat()

PiperOrigin-RevId: 510956177
2023-02-27 18:04:26 +00:00
samrobinson
9353f18ea0 Fix broken test due to file not found.
PiperOrigin-RevId: 512012099
2023-02-27 18:03:41 +00:00
huangdarwin
92138fc83c Effect: Rename to DefaultShaderProgram.
Rename:
* MatrixShaderProgram to DefaultShaderProgram, and
* FinalMatrixShaderProgramWrapper to FinalShaderProgramWrapper.
PiperOrigin-RevId: 510498547
2023-02-27 18:03:38 +00:00
huangdarwin
84acfe7867 Effect: Rename to ScaleAndRotateTransformation
Rename ScaleToFitTransformation to ScaleAndRotateTransformation.

This better represents the operations that can be accomplished using this
effect. The name was originally named ScaleToFit* because it's not obvious how
to scale to fit using OpenGL, and this effect handled the scaling to fit in a way that no other MatrixTransformations did.

However, it's hard to discover how to rotate when skimming names of effects, so
it's probably more useful to convey that this effect rotates, than that it
scales to fit.

PiperOrigin-RevId: 510480078
2023-02-27 18:02:45 +00:00
huangdarwin
f295dd4211 HDR: Add colorinfo to exception logs.
Format.toString unfortunately doesn't log colorInfo, and as Format holds a very
large set of values, it's unclear that it should. ColorInfo is useful for codec
exceptions though, so log this in ExportException.createForCodec.

PiperOrigin-RevId: 510475520
2023-02-27 18:01:53 +00:00
tofunmi
1308e22813 fix nits and enhance DefaultVideoFrameProcessorImageFrameOutputTest
resolving comments from 71e92f7512.

PiperOrigin-RevId: 510457401
2023-02-27 18:01:00 +00:00
kimvde
3f445a25d8 Throw when the video track disappears during export
Throw when the output has a video track but the current MediaItem in
the sequence doesn't have any video.

PiperOrigin-RevId: 512004463
2023-02-27 17:59:17 +00:00
bachinger
ba49b6b81b Map PLAYER_STATE_LOADING to STATE_BUFFERING
#minor-release
Issue: androidx/media#245
PiperOrigin-RevId: 510456793
2023-02-27 17:59:16 +00:00
tofunmi
71e92f7512 Add multi-image input FrameProcessor tests
PiperOrigin-RevId: 510441777
2023-02-27 17:58:25 +00:00
andrewlewis
b89b13f16b Align test codec caps check with real implementation
PiperOrigin-RevId: 510437262
2023-02-27 17:57:30 +00:00
sheenachhabra
f6155557eb Change DumpableBox to dump recursively
PiperOrigin-RevId: 510433206
2023-02-27 17:56:22 +00:00
kimvde
5806414fba Move generateSilentAudio to Composition
Also rename to forceAudioTrack

PiperOrigin-RevId: 510394620
2023-02-27 17:55:27 +00:00
Mayur K
0761641d9f Use edts media time offset if edits list duration is entire media duration. 2023-02-24 18:18:16 +05:30
andrewlewis
1ef70cd6a0 Add missing variant to package transformations
#minor-release

PiperOrigin-RevId: 512002735
2023-02-24 11:36:43 +00:00
tianyifeng
e91d1735b6 Detect BT devices when getting audio capabilities from the context
PiperOrigin-RevId: 511853003
2023-02-24 11:33:12 +00:00
samrobinson
511ad57de6 Propagate PCM encoding passed to AudioSamplePipeline.
PiperOrigin-RevId: 511810757
2023-02-24 11:29:35 +00:00
huangdarwin
285d9c9fe5 Test: Add format codec info and update HDR10 file to 720p.
Add format codec info, which can make test skipping checks more similar to the
actual Transformer decoder checks.

Also for the test file, the actual format was 720p, but somehow the file name and
media metadata indicated 1080p. This format mismatch led to some decoding errors,
so fix the format (and associated errors). This also allows us to remove the
exception catch in ForceInterpretHdrVideoAsSdrTest, which was included due to
errors from the incorrect format.

PiperOrigin-RevId: 511809507
2023-02-24 11:26:05 +00:00
tonihei
4e0babdcd1 Use ArrayDeque for pending output stream changes.
The current logic uses manual array operations to keep track of pending
changes. Modernize this code by using an ArrayDeque and a data class.
This also allows to extend the output stream information in the future.

This also fixes a bug where a position reset accidentally assigns a pending
stream offset instead of keeping the current one.

#minor-release

PiperOrigin-RevId: 511787571
2023-02-24 11:22:22 +00:00
kimvde
41a6d4aafb Clarify slow mo flattening Javadoc
PiperOrigin-RevId: 511778088
2023-02-24 11:18:50 +00:00
Googler
55b4c9a148 Update code to prepare for nullness annotations in Truth.
In most cases, this means updating Subject subclasses and their assertThat methods to accept null actual values. (They should always accept null so that assertions like "assertThat(foo).isNull()" succeed instead of throwing NullPointerException.)

Occasionally, it involves other changes, like writing `isGreaterThan(1L)` instead of `isGreaterThan(1)` to resolve an ambiguity it Kotlin overload resolution.

PiperOrigin-RevId: 511776581
2023-02-24 11:15:12 +00:00
kimvde
50f5e34583 Improve shouldTranscode() logic
PiperOrigin-RevId: 511767953
2023-02-24 11:11:40 +00:00
kimvde
c5c171beba Add unit tests for audio track dis(appearing) during export
PiperOrigin-RevId: 511764841
2023-02-24 11:08:07 +00:00
kimvde
4c9b8cb694 Document that slow mo flattening is only supported for single-asset
PiperOrigin-RevId: 511757712
2023-02-24 11:04:33 +00:00
kimvde
0205e75ff8 Split transmux into transmuxAudio/Video
- Split the transmux setting into transmuxAudio and transmuxVideo. This
  is more flexible for apps and will also be useful for unit testing
  (particularly as we can't test video transcoding on Robolectric at the
  moment).
- Move these settings to Composition. It makes sense for these settings
  to be next to forceAudioTrack. Apps may also want to set these
  settings based on the current Composition's MediaItems.
- Add a Composition.Builder because Composition now contains a few
  optional fields.

PiperOrigin-RevId: 511708618
2023-02-24 11:01:03 +00:00
samrobinson
7f3eb113e5 Fix disallowed tag.
If ChannelMixingMatrix is public, this causes code to not build.

PiperOrigin-RevId: 511500694
2023-02-24 10:57:22 +00:00
kimvde
379c3e1eb2 Restrict SampleConsumer and OnMediaItemChanged threading
This is more future-proof because it is likely to simplify the upcoming
changes to the sample pipelines.

PiperOrigin-RevId: 511492014
2023-02-24 10:53:53 +00:00
samrobinson
9a8026f27e Only log list of removed encoders if not empty.
Logcat had the following lines, with no other information.
```
DefaultEncoderFactory: Encoders removed for resolution:
DefaultEncoderFactory: Encoders removed for bitrate:
DefaultEncoderFactory: Encoders removed for bitrate mode:
```
PiperOrigin-RevId: 511470231
2023-02-24 10:50:20 +00:00
kimvde
12a6183312 Make sure that VideoSamplePipeline is picked for image input
PiperOrigin-RevId: 511449658
2023-02-24 10:46:49 +00:00
huangdarwin
cb7a8d2d03 Effect: Implement isNoOp on GlEffect interfaces.
Also, allow isNoOp to default to false without the TODO, so that implementations
of isNoOp must opt-in to implementing the override in order to be considered for
skipping the effect (ex. for transcoding in Transformer).

PiperOrigin-RevId: 511223540
2023-02-24 10:43:08 +00:00
sheenachhabra
17b3a2133b Move FtypAdjusterTest.java into muxer module
Changes include:
1. Move the test file into muxer module
2. Code improvement
2. Remove dependency from Mp4Slicer library

PiperOrigin-RevId: 511191076
2023-02-24 10:39:33 +00:00
kimvde
948ccfe2a6 Document that the output file is not deleted in case of error/cancel
PiperOrigin-RevId: 511189130
2023-02-24 10:36:05 +00:00
huangdarwin
7e377854a3 MediaFormatUtil: Reduce Color API 29 restriction to API 24.
Implement getMediaFormatInteger, a helper method simulating mediaformat.getInteger(name, defaultValue).

This reduces the API 29 restriction from MediaFormatUtil.getColorInfo to API 24, in
particular removing the method-based restriction to a constant-based restriction,
so that we can reduce usage of the API 29 class.

This also allows us to slightly simplify prior use-cases where we'd check
containsKey and getInteger to have a default value.

PiperOrigin-RevId: 511184301
2023-02-24 10:32:22 +00:00
kimvde
f4766ee4db Fix clipping in AudioSamplePipeline
When clipping a MediaItem with start time > 0, the audio was ending
before the video. This is because:
- Audio timestamps are computed based on the sample sizes, with a start
  time set to streamOffsetUs (i.e. the streamStartPositionUs is not
  taken into account).
- The SamplePipeline was subtracting streamStartPositionUs from the
  timestamps before sending the samples to the muxer.
- As a result, the audio timestamps were shifted by
  streamStartPositionUs, while they should be shifter by streamOffsetUs.

PiperOrigin-RevId: 511175923
2023-02-24 10:28:53 +00:00
kimvde
3009b4d5d1 Rename transformation to export in DebugViewProvider
PiperOrigin-RevId: 511173451
2023-02-24 10:25:18 +00:00
samrobinson
0344206814 Use the supported codec name in DecodeOneFrameUtil.
Tested:
  Before: fusion2/7d7db2d1-1c3e-3df3-8507-73b929c6eac8 (Weekly triage)
  After: sponge2/4c078211-e08c-43fa-91d5-346fccc04bcd (custom run)
PiperOrigin-RevId: 511157461
2023-02-24 10:21:38 +00:00
samrobinson
9ecab028a0 Expand and refactor TransformerInternal#shouldTranscode.
* Account for `AssetLoader` output types.
* Consider cases that are not audio/video specific.
* Use `Format#sampleMimeType` for track specific conditions to check.
* Untangle `SamplePipeline` initilization from `AssetLoader` state.

PiperOrigin-RevId: 511020865
2023-02-24 10:18:06 +00:00
kimvde
79d32c2488 Support added/removed audio track between MediaItems
- Add silent audio when the output contains an audio track but the
  current MediaItem doesn't have any audio.
- Add an audio track when generateSilentAudio is set to true.

PiperOrigin-RevId: 511005887
2023-02-24 10:14:26 +00:00
sheenachhabra
9fa8aba32e Add missing test cases for sample table boxes
Test cases for following boxes are added:
1. stsc
2. co64
3. stss

PiperOrigin-RevId: 511002841
2023-02-24 10:10:57 +00:00
huangdarwin
b206a1afec HDR: Fix ForceInterpretHdrVideoAsSdrTest, and expand to all APIs.
Previously, this was limited to API 29. Expand this to all API versions.

Also, update the test to:
(1) skip based on SDR format input instead of HDR format input
(2) Check the exception message in order to disambiguate between the decoder tone
mapping error, and general video format support error.

PiperOrigin-RevId: 511002218
2023-02-24 10:07:18 +00:00
samrobinson
43b8859cb7 Release the created placeholder surface in DecodeOneFrameUtil.
PiperOrigin-RevId: 511000498
2023-02-24 10:03:48 +00:00
tonihei
57a638aa88 Indicate that SCTE-35 metadata in not supported with HLS
Issue: google/ExoPlayer#10992

#minor-release

PiperOrigin-RevId: 510988140
2023-02-24 10:00:05 +00:00