437 Commits

Author SHA1 Message Date
Haixia Shi
834664be4f Add a unit test to Mp4ExtractorTest with a small av1 sample with color info. 2023-11-02 15:31:49 +00:00
microkatz
918079059c Format files and add release note 2023-11-02 15:31:49 +00:00
Haixia Shi
f455e843bc Address review comments on Av1BitstreamParser
* Rename method f(n) to a more helpful name.
* Move the private inner class to the bottom.
* Move public fields before private fields but below the static field.
* Make `private ParsableByteArray data` final.
* Make sure `parseSequenceHeader` can only be called once.
2023-11-02 15:31:48 +00:00
Haixia Shi
4ded7e588d Add AV1 color space parsing in MP4 atom parser
This adds av1C parsing based on the AV1 bitstream specification: https://aomediacodec.github.io/av1-spec/av1-spec.pdf

This is needed to have correct color space when using hardware AV1 decoder.
2023-11-02 15:31:48 +00:00
tonihei
d4e5ab2c4d Make BundleableUtil more generic by taking Function parameters
BundleableUtil is the only class that really depends on the type
inheritence from Bundleable. However, it only needs this as a way
to define Function<T, Bundle> and Function<Bundle, T>, which could
just be passed in as parameters as it's already done for some of
these methods.

Also rename the class to BundleCollectionUtil accordingly.

PiperOrigin-RevId: 578791946
2023-11-02 03:17:17 -07:00
ibaker
27caeb8038 Migrate CEA-608 logic to a SubtitleParser implementation
PiperOrigin-RevId: 578505919
2023-11-01 07:07:19 -07:00
jbibik
97efa70852 Add DASH support for parsing standalone TTML files during extraction
This change applies to standalone TTML files linked directly from the manifest.

As a result, we no longer have the flakiness in the DashPlaybackTest which uses sidecar-loaded (standalone file) TTML subtitles. We experimentally opt into parsing subtitles during extraction and use SubtitleExtractor in hybrid mode.

PiperOrigin-RevId: 577457256
2023-10-28 06:59:45 -07:00
Copybara-Service
009d48a75e Merge pull request #491 from v-novaltd:dsparano-exo128
PiperOrigin-RevId: 574129451
2023-10-17 06:35:19 -07:00
microkatz
212e138bcf Removed deprecated ColorInfo constructor call and fixed formatting 2023-10-17 11:42:33 +00:00
microkatz
d206be74ef Updated to use ColorInfo.Builder() and formatting issues 2023-10-17 11:42:33 +00:00
microkatz
c72ec74e11 Format with google-java-format 2023-10-17 11:42:33 +00:00
Daniele Sparano
f8d98861ea Move luma and chroma bitdepths to ColorInfo class 2023-10-17 11:42:33 +00:00
Daniele Sparano
59ff0dbb85 Improve isobmff atom parsing changes 2023-10-17 11:42:33 +00:00
Daniele Sparano
e71ca5a740 Add luma and chroma bit depths to Format, set them as signalled in containers and parameter sets 2023-10-17 11:42:32 +00:00
samrobinson
fe1144487a Parse metadata from srfr atom in SMTA.
PiperOrigin-RevId: 573829216
2023-10-16 09:11:53 -07:00
ibaker
ecd24646cb Migrate SubtitleParser implementations to incremental parse()
All production and test callers of the non-incremental methods are
already migrated, so we can remove them in this change too.

#minor-release

PiperOrigin-RevId: 573207318
2023-10-13 06:52:40 -07:00
ibaker
f9ece88a25 Change LegacySubtitleUtil handling of SubtitleParser.OutputOptions
If the `Subtitle` has 'active' cues at `OutputOptions.startTimeUs`, this
change ensures these are emitted in a `CuesWithTiming` with
`CuesWithTiming.startTimeUs = OutputOptions.startTimeUs`. If
`OutputOptions.outputAllCues` is also set, then another `CuesWithTiming`
is emitted at the end that covers the 'first part' of the active cues,
and  ends at `OutputOptions.startTimeUs`.

As well as adding some more tests to `LegacySubtitleUtilWebvttTest`,
this change also adds more tests for `TtmlParser` handling of
`OutputOptions`, which transitively tests the behaviour of
`LegacySubtitleUtil`.

#minor-release

PiperOrigin-RevId: 573151016
2023-10-13 02:21:22 -07:00
ibaker
a12bde4f57 Migrate SubtitleParser tests to incremental parse() methods
All the production code is already calling these new incremental
methods, migrating the tests allows us to remove the old
`List`-returning methods in a follow-up change.

#minor-release

PiperOrigin-RevId: 572822828
2023-10-12 01:42:28 -07:00
ibaker
164e658839 Test that DefaultSubtitleParserFactory format support is consistent
Looping through all MIME types is a bit hacky, but seems like a nice way
to check "all" MIME types (or at least more than a random selection).

PiperOrigin-RevId: 572508027
2023-10-11 02:10:30 -07:00
ibaker
66fa591959 Add experimental opt-in to parse DASH subtitles during extraction
This currently only applies to subtitles muxed into mp4 segments, and
not standalone text files linked directly from the manifest.

Issue: androidx/media#288

#minor-release

PiperOrigin-RevId: 572263764
2023-10-10 08:53:42 -07:00
ibaker
e366c3d419 Add SubtitleParser.Factory.getCueReplacementBehavior()
This gives access to the replacement behavior for a particular subtitle
format without needing to instantiate a `SubtitleParser`.

#minor-release

PiperOrigin-RevId: 572226084
2023-10-10 06:20:08 -07:00
ibaker
3ed7e561bb Set Format.cueReplacementBehavior in SubtitleExtractor
This is more correct, though in reality all text-based subtitles handled
by `SubtitleExtractor` are implemented with MERGE behavior in media3.

PiperOrigin-RevId: 571888573
2023-10-09 03:31:40 -07:00
tonihei
89d01981bc Deprecate decode-only flag.
The flag is no longer used by our components and only set and checked
in a few places to guarantee compatiblity with existing renderers and
decoders that still use it.

The flag will be removed in the future due to its design limitations.

#minor-release

PiperOrigin-RevId: 571291168
2023-10-06 03:44:14 -07:00
ibaker
002ee0555d Update TextRenderer to handle CuesWithTiming instances directly
The existing `Subtitle` handling code is left intact to support the
legacy post-`SampleQueue` decoding path for now.

This also includes full support for merging overlapping `CuesWithTiming`
instances, which explains the test dump file changes, and which should
resolve the following issues (if used with the
decoder-before-`SampleQueue` subtitle logic added in
5d453fcf37):

* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794

It should also help resolve Issue: androidx/media#288, but that will also require
some changes in the DASH module to enable pre-`SampleQueue` subtitle
parsing (which should happen soon).

#minor-release

PiperOrigin-RevId: 571021417
2023-10-05 08:17:23 -07:00
ibaker
6057b59723 Add tests for CuesWithTiming.endTimeUs
#minor-release

PiperOrigin-RevId: 570988195
2023-10-05 05:51:54 -07:00
ibaker
bf7b91e57e Add CuesWithTiming.endTimeUs
In most cases this is more useful than `durationUs`.

We will keep `durationUs`, and the constructor will continue to take
`startTimeUs` and `durationUs`, to allow for use-cases where we don't
know the start time but still want to indicate a duration (this will be
used to implement CEA-608 timeout).

#minor-release

PiperOrigin-RevId: 570944449
2023-10-05 02:04:26 -07:00
tonihei
a03e20fe6c Add Decoder.setOutputStartTimeUs and use it in extension decoders
This gets rid of the reliance on the decode only flag that is still
set on input buffers to the decoder if they are less than the start
time.

We still need to set and check the decode-only flag in SimpleDecoder
to ensure compatbility with custom decoders that use the flag while
it's not fully removed.

PiperOrigin-RevId: 570736692
2023-10-04 10:37:25 -07:00
tonihei
64fe863f31 Remove wrong Javadoc
The corresponding logic has been removed in 796781d4c3

#minor-release

PiperOrigin-RevId: 570729509
2023-10-04 10:13:12 -07:00
tonihei
796781d4c3 Move decode-only handling out of MetadataDecoder interface logic
The interface requires the implementation to return null if the
decode-only flag is set. So instead of setting the flag and returning
null, we can simply not call the method and assume it's null.

The only reason where this wouldn't work is if the metadata format
has keyframe-like logic and requires previous metadata to decode
the next one. This is not something we came across before and it seems
ignorable. If that feature is needed in the future, we should instead
add a method to MetadataDecoder to set the first output timestamp.

#minor-release

PiperOrigin-RevId: 570399838
2023-10-03 08:49:18 -07:00
ibaker
134f53bdb2 TTML: Remove unused CellResolution.columns and collapse to int rows
PiperOrigin-RevId: 566908824
2023-09-20 03:22:21 -07:00
ibaker
694d040d7e Re-land: TtmlParser implementation - moved from TtmlDecoder
This change uses the new incremental overloads of `SubtitleParser` to
avoid introducing the performance regression caused by the original
change requiring all cues to be fully parsed before the first could be
shown on screen.

`TtmlDecoder` which used to be `SimpleSubtitleDecoder` will now be
called `TtmlParser` and implement `SubtitleParser` interface. For
backwards compatibility, we will have the same functionality provided
by `DelegatingSubtitleDecoder` backed-up by a new `TtmlParser`
instance.

PiperOrigin-RevId: 566671398
2023-09-19 10:25:48 -07:00
ibaker
f35e7bdc6a Switch WebvttParserTest to use partial-output method
PiperOrigin-RevId: 566598735
2023-09-19 05:19:07 -07:00
ibaker
92a3f3a8cd Update WebvttParser to implement new partial-output method
This requires adapting the 'to `CuesWithTiming` list' logic to work with
the new partial-output API, and that needs a private method so it's no
longer a good fit for a default method on `Subtitle` - hence moving it
to a new utility class.

Also update the implementation to never return `UNSET` duration (this is
an equivalent change to the `SsaParser` change in 9631923440).

PiperOrigin-RevId: 566598094
2023-09-19 05:15:08 -07:00
rohks
fe199455e7 Add QuickTime (Classic) support to Mp4Extractor
PiperOrigin-RevId: 566272132
2023-09-18 05:29:29 -07:00
ibaker
56fbf7377e Add javadoc to SubtitleParser.OutputOptions
PiperOrigin-RevId: 565066243
2023-09-13 09:04:27 -07:00
ibaker
d111976125 Change SubtitleParser interface to support incremental output
This change introduces two new types of method to `SubtitleParser`:
1. `parse()` methods that take a `Consumer<CuesWithTiming>` and return `void`
2. `parseToLegacySubtitle` method that returns `Subtitle`

(1) ensures that in the new 'parse before SampleQueue' world we can
write cues to the `SampleQueue` as soon as they're ready - this is
especially important when parsing monolithic text files, e.g. for a
whole movie.

(2) ensures that during the transition, the legacy 'parse after
SampleQueue' behaviour doesn't see any regressions in 'time to first
cue being shown'. Previously we had a single implementation to convert
from `List<CuesWithTiming>` to `Subtitle`, but this relies on the
complete list of cues being available, which can take a long time for
large files in some formats (with ExoPlayer's current parsing logic).
By allowing implementations to customise the way they create a
`Subtitle`, we can directly re-use the existing logic, so that the
'time to first cue being shown' should stay the same.

This change migrates all **usages** to the new methods, but doesn't
migrate any **implementations**. I will migrate the implementations in
follow-up CLs before deleting the old list-returning `parse()` methods.

PiperOrigin-RevId: 565057945
2023-09-13 08:33:29 -07:00
ibaker
fdfec60f6e Remove some debugging lines in AtomParsers
These were accidentally added in 885ddb167e

PiperOrigin-RevId: 565035337
2023-09-13 06:49:25 -07:00
ibaker
abaf3e7aa1 Add Format.cueReplacementBehavior
Previously any `CuesWithTiming.durationUs` could be `TIME_UNSET`,
meaning it should be replaced by the next `CuesWithTiming` instance
(instead of being merged if the durations overlap, which is currently
the expected behavior for all `CuesWithTiming` with a 'real' duration).

This technically allowed a single subtitle track to include a mixture of
`CuesWithTiming` that should be merged, and some that should be
replaced. This is not actually needed for any of the subtitle formats
currently supported by ExoPlayer - in all cases a format expects either
all cues to be merged, or each cue to replace the previous one.

Supporting this mixture of merging and replacing in `TextRenderer` ended
up being very complicated, and it seemed a bit pointless since it's not
actually needed. This change means a given subtitle track either merges
**all** cues (meaning `CuesWithTiming.durationUs = C.TIME_UNSET` is not
allowed), or **every** cue is replaced by the next one (meaning
`CuesWithTiming.durationUs` may be set (to allow for cues to 'time out',
needed for CEA-608), or may be `TIME_UNSET`).

This value will be used in a subsequent change that adds cue-merging
support to `TextRenderer`.

PiperOrigin-RevId: 565028066
2023-09-13 06:09:45 -07:00
ibaker
885ddb167e Add general-purpose overflow-resistant divide+multiply util method
This is equivalent to the existing `scaleLargeTimestamp` method with the
following changes/improvements:
* No longer specific to timestamps (there was nothing inherently
  time-specific about the logic in `scaleLargeTimestamp`, but the name
  and docs suggested it shouldn't be used for non-timestamp use-cases).
* Additional 'perfect division' checks between `value` and `divisor`.
* The caller can now provide a `RoundingMode`.
* Robust against `multiplier == 0`.
* Some extra branches before falling through to (potentially lossy)
  floating-point math, including trying to simplify the fraction with
  greatest common divisor to reduce the chance of overflowing `long`.

This was discussed during review of 6e91f0d4c5

This change also includes some golden test file updates - these
represent a bug fix where floating-point maths had previously resulted
in a timestamp being incorrectly rounded down to the previous
microsecond. These changes are due to the 'some more branches' mentioned
above.

PiperOrigin-RevId: 564760748
2023-09-12 10:22:57 -07:00
tofunmi
23665090ed Support Jpeg image track extraction in exoplayer
PiperOrigin-RevId: 563818198
2023-09-08 12:15:10 -07:00
tofunmi
be5fa6d130 Support image track extraction for JPEG
PiperOrigin-RevId: 563746945
2023-09-08 07:32:10 -07:00
tofunmi
535af35511 Add heif extractor
PiperOrigin-RevId: 563340591
2023-09-07 00:31:04 -07:00
tofunmi
88b114a997 Add bmpExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 563076972
2023-09-06 05:41:43 -07:00
tofunmi
45e15d48c4 Add WebpExtractor to DefaultExtractorsFactory
PiperOrigin-RevId: 563063506
2023-09-06 04:26:53 -07:00
tofunmi
a1be9d5a60 Add WebpExtractor
Also adds it to the DefaultExtractorsFactory for use throughout exoplayer

PiperOrigin-RevId: 562784760
2023-09-05 08:23:03 -07:00
tofunmi
9b2668c0ca Add BmpExtractor
Takes single sample extraction logic out of the png extractor and puts it in a helper so it can be used for bmp as well.

PiperOrigin-RevId: 562581815
2023-09-04 10:37:28 -07:00
michaelkatz
9d58788d86 Add AudioSink#setOffloadDelayPadding to offload play gapless Opus
Created new method `AudioSink#setOffloadDelayPadding` that will set delay and padding data onto an `AudioTrack` using `AudioTrack#setOffloadDelayPadding`. This feature adds support for offloaded, gapless Opus playback as the content requires the capability to set padding data post-`AudioSink#configure`.

PiperOrigin-RevId: 562518193
2023-09-04 04:02:44 -07:00
ibaker
8af11a980a Remove duplication of various TestUtil methods
In most cases, the places that were trying to avoid a transitive
dependency on `lib-exoplayer` when this duplication was introduced [1]
are already depending on it again, except for `lib-container` where the
dep is added in this change.

In general it seems fine for the tests of module A to depend
(transitively or directly) on module B even where the prod code of
module A **does not** depend on module B.

[1] <unknown commit>

PiperOrigin-RevId: 561660371
2023-08-31 08:23:33 -07:00
ibaker
64792dfda2 Rollback of 25253698bc
PiperOrigin-RevId: 561352515
2023-08-30 08:53:49 -07:00
ibaker
9ee45fc938 Add TrackOutput field in PngExtractor
Also re-jig some `checkNotNull` calls to satisfy the nullness checker
(remove an unneeded one, and add a new one).

Follow-up to discussion in bb214b19f9

PiperOrigin-RevId: 561295724
2023-08-30 04:08:56 -07:00