309 Commits

Author SHA1 Message Date
Daniele Sparano
4a17ca7600 Add basic h264 sample unit test to ensure correct sample flags, including key frames 2023-12-11 14:28:11 +00:00
Daniele Sparano
d4af13803a Fix H264 reader key frame detection 2023-12-11 14:28:11 +00:00
ibaker
379cb3ba54 Tighten the handling of validDataChannelTimeoutMs in Cea608Parser
This matches the documented requirements for this parameter.

PiperOrigin-RevId: 589799256
2023-12-11 06:05:53 -08:00
ibaker
bd19953ac9 Split Cue.toBundle into serializable and binder-based variants
The serializable form is used when we need to serialize the result into
bytes in the sample queue. The binder-based (ultimately
filedescriptor-based) form is used for
session/controller IPC, in order to avoid sending the bitmap bytes over
the IPC.

Issue: androidx/media#836

#minor-release

PiperOrigin-RevId: 588420836
2023-12-06 07:35:34 -08:00
ibaker
6360082b87 Add extractor and playback tests for Pixel JPEG motion photo
This image has two video tracks in the MP4 data, one is a 'real' video
which we want to play by default, and the other is a low-fps video track
which isn't intended to be directly played, it's encoded in HEVC for
compression and decoding efficiency.

This test demonstrates ExoPlayer's current behaviour default extraction
and playback, which results in selecting the high-res, low-fps track
(actually single sample in this example), instead of playing the actual
video.

PiperOrigin-RevId: 588068908
2023-12-05 07:46:19 -08:00
michaelkatz
d1e38abf93 Limit processing Opus decode-only frames by seek-preroll in offload
As Opus decoders skip some bytes prior to playback during a seek, the renderer for bypass playback should send samples to the decoder even if they would be decode-only. However, the renderer should not send samples with time preceding that range. This change adds that constraint.

#minor-release

PiperOrigin-RevId: 588014983
2023-12-05 03:44:37 -08:00
tonihei
b1541b096f Map VORBIS channel layout to Android layout
Both the extension OPUS decoder and the OMX/C2 MediaCodec
implementations for OPUS and VORBIS decode into the channel
layout defined by VORBIS. See
https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-140001.2.3

While this is technically correct for a stand-alone OPUS or VORBIS
decoder, it doesn't match the channel layout expected by Android.
See https://developer.android.com/reference/android/media/AudioFormat#channelMask

The fix is to apply the channel mapping after decoding if needed.
Also add e2e tests with audio dumps for the extension renderer,
including a new 5.1 channel test file.

Issue: google/ExoPlayer#8396

#minor-release

PiperOrigin-RevId: 588004832
2023-12-05 03:01:32 -08:00
ibaker
63062a9c10 Make CueSerializationTest more realistic
Serializing bitmap cues is currently broken, but this test is
incorrectly passing. This change makes two changes to introduce the same
failure (both changes are necessary, each one alone still passes):

1. Move from Robolectric to an instrumentation test.
2. Trigger the `Bitmap` to be serialized using a file descriptor, either
   by calling `Bitmap.asShared` in the test when constructing the `Cue`,
   or constructing the `Bitmap` from a 'real' image byte array instead a
   1x1 token image.

Issue: androidx/media#836
PiperOrigin-RevId: 585643486
2023-11-27 07:16:19 -08:00
jbibik
c0ef5f6de4 Serialize media3 custom Spans for Cue encoding/decoding
PiperOrigin-RevId: 585028521
2023-11-24 01:32:40 -08:00
Copybara-Service
fde142d66e Merge pull request #793 from androidx:mpegh_extractor_changes
PiperOrigin-RevId: 584291984
2023-11-21 05:07:27 -08:00
Rohit Singh
4c3ca9b6f4 update mhm1 mp4 samples and corresponding extractor dumps 2023-11-21 12:12:23 +00:00
ibaker
4bcb60d31d Add Cea708Decoder/ParserTest with a single, simple example
PiperOrigin-RevId: 584042033
2023-11-20 09:06:03 -08:00
jbibik
94e45eb4ad Migrate CEA-708 logic to a SubtitleParser implementation
This change is similar to 27caeb8038 (CEA-608), but is currently lacking unit tests.

PiperOrigin-RevId: 582585457
2023-11-15 01:38:56 -08:00
tonihei
b570c72588 Normalize MIME types when accepting user or media input
MIME types are case-insensitive, but none of the many existing
comparisons across our code base take this into account. The
code can be made more robust by normalizing all MIME types at the
moment they are first set into a class/builder and adding toLowerCase
as part of the normalization.

Most concretely, this fixes an issue with playing HLS streams via
the IMA SDK where the stream MIME type is indicated with all lower
case "application/x-mpegurl", which failed the MIME type comparison
in DefaultMediaSourceFactory.

PiperOrigin-RevId: 582317261
2023-11-14 07:40:15 -08:00
Rohit Singh
6297e7aeae Changes based on internal review 2023-11-07 14:23:07 +00:00
Rohit Singh
8e554c6d6a - some further changes according to review comments
- update MPEG-H test data and extractor dumps
2023-11-07 13:01:20 +00:00
Rohit Singh
6d2059080b - changes according to review comments
- add test data and dumps
2023-11-06 13:28:26 +00:00
Rohit Singh
94442291bd Extend MP4 extractor implementation with complete parsing of mhaC and mhaP boxes 2023-11-06 13:11:46 +00:00
microkatz
dc530289bb Changed av1c parser from a class to a method 2023-11-02 15:38:27 +00:00
microkatz
9d43cdac17 Add av1C to atom list comment in colr atom parse 2023-11-02 15:31:49 +00:00
Haixia Shi
258f45f874 Fix unit test problems with sample_with_av1c.mp4 2023-11-02 15:31:49 +00:00
microkatz
5a70bbfeec Format with google-java-format 2023-11-02 15:31:49 +00:00
Haixia Shi
834664be4f Add a unit test to Mp4ExtractorTest with a small av1 sample with color info. 2023-11-02 15:31:49 +00:00
microkatz
918079059c Format files and add release note 2023-11-02 15:31:49 +00:00
Haixia Shi
f455e843bc Address review comments on Av1BitstreamParser
* Rename method f(n) to a more helpful name.
* Move the private inner class to the bottom.
* Move public fields before private fields but below the static field.
* Make `private ParsableByteArray data` final.
* Make sure `parseSequenceHeader` can only be called once.
2023-11-02 15:31:48 +00:00
Haixia Shi
4ded7e588d Add AV1 color space parsing in MP4 atom parser
This adds av1C parsing based on the AV1 bitstream specification: https://aomediacodec.github.io/av1-spec/av1-spec.pdf

This is needed to have correct color space when using hardware AV1 decoder.
2023-11-02 15:31:48 +00:00
tonihei
d4e5ab2c4d Make BundleableUtil more generic by taking Function parameters
BundleableUtil is the only class that really depends on the type
inheritence from Bundleable. However, it only needs this as a way
to define Function<T, Bundle> and Function<Bundle, T>, which could
just be passed in as parameters as it's already done for some of
these methods.

Also rename the class to BundleCollectionUtil accordingly.

PiperOrigin-RevId: 578791946
2023-11-02 03:17:17 -07:00
ibaker
27caeb8038 Migrate CEA-608 logic to a SubtitleParser implementation
PiperOrigin-RevId: 578505919
2023-11-01 07:07:19 -07:00
jbibik
97efa70852 Add DASH support for parsing standalone TTML files during extraction
This change applies to standalone TTML files linked directly from the manifest.

As a result, we no longer have the flakiness in the DashPlaybackTest which uses sidecar-loaded (standalone file) TTML subtitles. We experimentally opt into parsing subtitles during extraction and use SubtitleExtractor in hybrid mode.

PiperOrigin-RevId: 577457256
2023-10-28 06:59:45 -07:00
Copybara-Service
009d48a75e Merge pull request #491 from v-novaltd:dsparano-exo128
PiperOrigin-RevId: 574129451
2023-10-17 06:35:19 -07:00
microkatz
212e138bcf Removed deprecated ColorInfo constructor call and fixed formatting 2023-10-17 11:42:33 +00:00
microkatz
d206be74ef Updated to use ColorInfo.Builder() and formatting issues 2023-10-17 11:42:33 +00:00
microkatz
c72ec74e11 Format with google-java-format 2023-10-17 11:42:33 +00:00
Daniele Sparano
f8d98861ea Move luma and chroma bitdepths to ColorInfo class 2023-10-17 11:42:33 +00:00
Daniele Sparano
59ff0dbb85 Improve isobmff atom parsing changes 2023-10-17 11:42:33 +00:00
Daniele Sparano
e71ca5a740 Add luma and chroma bit depths to Format, set them as signalled in containers and parameter sets 2023-10-17 11:42:32 +00:00
samrobinson
fe1144487a Parse metadata from srfr atom in SMTA.
PiperOrigin-RevId: 573829216
2023-10-16 09:11:53 -07:00
ibaker
ecd24646cb Migrate SubtitleParser implementations to incremental parse()
All production and test callers of the non-incremental methods are
already migrated, so we can remove them in this change too.

#minor-release

PiperOrigin-RevId: 573207318
2023-10-13 06:52:40 -07:00
ibaker
f9ece88a25 Change LegacySubtitleUtil handling of SubtitleParser.OutputOptions
If the `Subtitle` has 'active' cues at `OutputOptions.startTimeUs`, this
change ensures these are emitted in a `CuesWithTiming` with
`CuesWithTiming.startTimeUs = OutputOptions.startTimeUs`. If
`OutputOptions.outputAllCues` is also set, then another `CuesWithTiming`
is emitted at the end that covers the 'first part' of the active cues,
and  ends at `OutputOptions.startTimeUs`.

As well as adding some more tests to `LegacySubtitleUtilWebvttTest`,
this change also adds more tests for `TtmlParser` handling of
`OutputOptions`, which transitively tests the behaviour of
`LegacySubtitleUtil`.

#minor-release

PiperOrigin-RevId: 573151016
2023-10-13 02:21:22 -07:00
ibaker
a12bde4f57 Migrate SubtitleParser tests to incremental parse() methods
All the production code is already calling these new incremental
methods, migrating the tests allows us to remove the old
`List`-returning methods in a follow-up change.

#minor-release

PiperOrigin-RevId: 572822828
2023-10-12 01:42:28 -07:00
ibaker
164e658839 Test that DefaultSubtitleParserFactory format support is consistent
Looping through all MIME types is a bit hacky, but seems like a nice way
to check "all" MIME types (or at least more than a random selection).

PiperOrigin-RevId: 572508027
2023-10-11 02:10:30 -07:00
ibaker
66fa591959 Add experimental opt-in to parse DASH subtitles during extraction
This currently only applies to subtitles muxed into mp4 segments, and
not standalone text files linked directly from the manifest.

Issue: androidx/media#288

#minor-release

PiperOrigin-RevId: 572263764
2023-10-10 08:53:42 -07:00
ibaker
e366c3d419 Add SubtitleParser.Factory.getCueReplacementBehavior()
This gives access to the replacement behavior for a particular subtitle
format without needing to instantiate a `SubtitleParser`.

#minor-release

PiperOrigin-RevId: 572226084
2023-10-10 06:20:08 -07:00
ibaker
3ed7e561bb Set Format.cueReplacementBehavior in SubtitleExtractor
This is more correct, though in reality all text-based subtitles handled
by `SubtitleExtractor` are implemented with MERGE behavior in media3.

PiperOrigin-RevId: 571888573
2023-10-09 03:31:40 -07:00
tonihei
89d01981bc Deprecate decode-only flag.
The flag is no longer used by our components and only set and checked
in a few places to guarantee compatiblity with existing renderers and
decoders that still use it.

The flag will be removed in the future due to its design limitations.

#minor-release

PiperOrigin-RevId: 571291168
2023-10-06 03:44:14 -07:00
ibaker
002ee0555d Update TextRenderer to handle CuesWithTiming instances directly
The existing `Subtitle` handling code is left intact to support the
legacy post-`SampleQueue` decoding path for now.

This also includes full support for merging overlapping `CuesWithTiming`
instances, which explains the test dump file changes, and which should
resolve the following issues (if used with the
decoder-before-`SampleQueue` subtitle logic added in
5d453fcf37):

* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794

It should also help resolve Issue: androidx/media#288, but that will also require
some changes in the DASH module to enable pre-`SampleQueue` subtitle
parsing (which should happen soon).

#minor-release

PiperOrigin-RevId: 571021417
2023-10-05 08:17:23 -07:00
ibaker
6057b59723 Add tests for CuesWithTiming.endTimeUs
#minor-release

PiperOrigin-RevId: 570988195
2023-10-05 05:51:54 -07:00
ibaker
bf7b91e57e Add CuesWithTiming.endTimeUs
In most cases this is more useful than `durationUs`.

We will keep `durationUs`, and the constructor will continue to take
`startTimeUs` and `durationUs`, to allow for use-cases where we don't
know the start time but still want to indicate a duration (this will be
used to implement CEA-608 timeout).

#minor-release

PiperOrigin-RevId: 570944449
2023-10-05 02:04:26 -07:00
tonihei
a03e20fe6c Add Decoder.setOutputStartTimeUs and use it in extension decoders
This gets rid of the reliance on the decode only flag that is still
set on input buffers to the decoder if they are less than the start
time.

We still need to set and check the decode-only flag in SimpleDecoder
to ensure compatbility with custom decoders that use the flag while
it's not fully removed.

PiperOrigin-RevId: 570736692
2023-10-04 10:37:25 -07:00
tonihei
64fe863f31 Remove wrong Javadoc
The corresponding logic has been removed in 796781d4c3

#minor-release

PiperOrigin-RevId: 570729509
2023-10-04 10:13:12 -07:00