267 Commits

Author SHA1 Message Date
ibaker
b77a2530af Change LegacySubtitleUtil handling of SubtitleParser.OutputOptions
If the `Subtitle` has 'active' cues at `OutputOptions.startTimeUs`, this
change ensures these are emitted in a `CuesWithTiming` with
`CuesWithTiming.startTimeUs = OutputOptions.startTimeUs`. If
`OutputOptions.outputAllCues` is also set, then another `CuesWithTiming`
is emitted at the end that covers the 'first part' of the active cues,
and  ends at `OutputOptions.startTimeUs`.

As well as adding some more tests to `LegacySubtitleUtilWebvttTest`,
this change also adds more tests for `TtmlParser` handling of
`OutputOptions`, which transitively tests the behaviour of
`LegacySubtitleUtil`.

#minor-release

PiperOrigin-RevId: 573151016
(cherry picked from commit f9ece88a25b449ab1e59ec0f6a67b71d7a2dc8ce)
2023-10-23 13:14:42 +01:00
ibaker
03a3f77340 Add experimental opt-in to parse DASH subtitles during extraction
This currently only applies to subtitles muxed into mp4 segments, and
not standalone text files linked directly from the manifest.

Issue: androidx/media#288

#minor-release

PiperOrigin-RevId: 572263764
(cherry picked from commit 66fa5919590789b384506a4e604fe02a5a5e0877)
2023-10-23 13:14:41 +01:00
ibaker
99ea703455 Fix the asset and dump file names for the standalone TTML DASH test
#minor-release

PiperOrigin-RevId: 571941997
(cherry picked from commit 33c151eb5b5ce85e554215af0c1d860b66c66fab)
2023-10-23 13:13:46 +01:00
ibaker
6ad54d72ad Update TextRenderer to handle CuesWithTiming instances directly
The existing `Subtitle` handling code is left intact to support the
legacy post-`SampleQueue` decoding path for now.

This also includes full support for merging overlapping `CuesWithTiming`
instances, which explains the test dump file changes, and which should
resolve the following issues (if used with the
decoder-before-`SampleQueue` subtitle logic added in
5d453fcf37):

* Issue: google/ExoPlayer#10295
* Issue: google/ExoPlayer#4794

It should also help resolve Issue: androidx/media#288, but that will also require
some changes in the DASH module to enable pre-`SampleQueue` subtitle
parsing (which should happen soon).

PiperOrigin-RevId: 571021417
(cherry picked from commit 002ee0555dc35dce9570f1a991b33ec92743db10)
2023-10-23 13:13:42 +01:00
tonihei
7e9ca0bf74 Deprecate decode-only flag.
The flag is no longer used by our components and only set and checked
in a few places to guarantee compatiblity with existing renderers and
decoders that still use it.

The flag will be removed in the future due to its design limitations.

#minor-release

PiperOrigin-RevId: 571291168
(cherry picked from commit 89d01981bc4cf9218e73bcce1b52c7afe29fbecd)
2023-10-06 15:57:25 +01:00
jbibik
e4637d82b7 Add DashPlayback test with sideloaded TTML subtitles
The test is hidden behind the Ignore annotation due to some flakiness just like `webvttInMp4`. However, it will be removed when the subtitle parsing is moved to a pre-sample-queue architecture.

#minor-release

PiperOrigin-RevId: 570376275
(cherry picked from commit bd5a3920b85423f1c1b625369fd606a9f36e1248)
2023-10-06 15:57:24 +01:00
christosts
0b4638af15 ExportTest: make 8K asset and trim
Move remote 8K file to local and trim to 320ms.

Trim done with ffmpeg:

`ffmpeg -i {remote_file} -t 0.3 -c:v copy -c:a copy 8k24fps_300ms.mp4`

PiperOrigin-RevId: 569449962
2023-09-29 04:07:40 -07:00
michaelkatz
c12fb67468 Update dumpfile tests to only print ColorInfo fields that are set
PiperOrigin-RevId: 568789489
2023-09-27 02:34:29 -07:00
jbibik
8b71712edc Change the DASH playback dump for webvtt in mp4 to include subtitles
#minor-release

PiperOrigin-RevId: 568567703
2023-09-26 09:48:52 -07:00
huangdarwin
ba8c85a277 Compositor: Add VideoCompositorSettings to Composition.
This allows apps using Transformer to customize how a Composition is used.

PiperOrigin-RevId: 567633129
2023-09-22 08:51:35 -07:00
rohks
fe199455e7 Add QuickTime (Classic) support to Mp4Extractor
PiperOrigin-RevId: 566272132
2023-09-18 05:29:29 -07:00
tofunmi
16b0ea850f Support multiple streams in the ImageRenderer
PiperOrigin-RevId: 565410924
2023-09-14 10:37:58 -07:00
claincly
1d8135e563 Add test for MultiInputVideoGraph
This test composites the first frame from two video inputs.

PiperOrigin-RevId: 565090338
2023-09-13 10:28:07 -07:00
ibaker
885ddb167e Add general-purpose overflow-resistant divide+multiply util method
This is equivalent to the existing `scaleLargeTimestamp` method with the
following changes/improvements:
* No longer specific to timestamps (there was nothing inherently
  time-specific about the logic in `scaleLargeTimestamp`, but the name
  and docs suggested it shouldn't be used for non-timestamp use-cases).
* Additional 'perfect division' checks between `value` and `divisor`.
* The caller can now provide a `RoundingMode`.
* Robust against `multiplier == 0`.
* Some extra branches before falling through to (potentially lossy)
  floating-point math, including trying to simplify the fraction with
  greatest common divisor to reduce the chance of overflowing `long`.

This was discussed during review of 6e91f0d4c5

This change also includes some golden test file updates - these
represent a bug fix where floating-point maths had previously resulted
in a timestamp being incorrectly rounded down to the previous
microsecond. These changes are due to the 'some more branches' mentioned
above.

PiperOrigin-RevId: 564760748
2023-09-12 10:22:57 -07:00
tofunmi
23665090ed Support Jpeg image track extraction in exoplayer
PiperOrigin-RevId: 563818198
2023-09-08 12:15:10 -07:00
tofunmi
be5fa6d130 Support image track extraction for JPEG
PiperOrigin-RevId: 563746945
2023-09-08 07:32:10 -07:00
samrobinson
866d62dd34 Improve CompositionExportTest assertions by using dump files.
PiperOrigin-RevId: 563708666
2023-09-08 04:09:21 -07:00
samrobinson
763dddfbd4 Dump with C.TrackType as key, rather than muxer track index.
Modifying dumping to not required "released" to be called.

Track index is an arbitrary value based on the order of addTrack calls.
Samples are dumped by track (rather than as soon as they are written),
so it's preferable to use a value that provides more context.

By using the track type as a key, dump files will be more deterministic
and will have more similarities when branched.

PiperOrigin-RevId: 563700982
2023-09-08 03:25:13 -07:00
tofunmi
535af35511 Add heif extractor
PiperOrigin-RevId: 563340591
2023-09-07 00:31:04 -07:00
samrobinson
cff2816da4 Add silence generation parameterized test case.
PiperOrigin-RevId: 563098931
2023-09-06 07:34:32 -07:00
samrobinson
92814b84a8 Add alternate MP4 asset with PCM audio.
Audio is Mono 44.1kHz.

Created using:
`ffmpeg -i <input> -c:v copy -c:a pcm_s16be <output>`

PiperOrigin-RevId: 563079553
2023-09-06 05:58:27 -07:00
samrobinson
ef45c0fe5d Add parameterized dump tests for single item exports.
PiperOrigin-RevId: 563075771
2023-09-06 05:35:51 -07:00
tofunmi
a1be9d5a60 Add WebpExtractor
Also adds it to the DefaultExtractorsFactory for use throughout exoplayer

PiperOrigin-RevId: 562784760
2023-09-05 08:23:03 -07:00
sheenachhabra
657856dfe5 Fix/Update TODOs in Mp4Muxer tests
PiperOrigin-RevId: 562728777
2023-09-05 04:04:51 -07:00
tofunmi
9b2668c0ca Add BmpExtractor
Takes single sample extraction logic out of the png extractor and puts it in a helper so it can be used for bmp as well.

PiperOrigin-RevId: 562581815
2023-09-04 10:37:28 -07:00
christosts
70ad637e52 Conditionally output captured data in CapturingRenderersFactory
This change makes capturing components in CapturingRenderersFactory
to dump data only they have previously captured any. This is so we can
extend the CapturingRenderersFactory with more renderers that do not
capture data in pre-existing tests and we don't have to change the
golden files.

PiperOrigin-RevId: 561973645
2023-09-01 09:26:33 -07:00
tofunmi
e15c05fc3c Flip input buffer in the ImageRenderer
also improves tests to hash bitmap now that we know the bitmaps are decoded in roboelectic tests,
as well as some minor fixes in the DefaultImageDecoder

PiperOrigin-RevId: 561946073
2023-09-01 07:12:57 -07:00
huangdarwin
57bc215210 Compositor: Implement OverlaySettings and custom in/out size support.
Implement VideoCompositor support of:
* Different input and output sizes
* CompositorSettings, to customize output size based on input texture sizes
* OverlaySettings, to place an input frame in an arbitrary position on
  the output frame.

Also, refactor Overlay's matrix logic to make it more reusable between
Compositor and Overlays

PiperOrigin-RevId: 561931854
2023-09-01 05:54:15 -07:00
tofunmi
5944adcc78 Add image e2e test
PiperOrigin-RevId: 561887238
2023-09-01 01:31:45 -07:00
tofunmi
bb214b19f9 Sample metadata in the png extractor
PiperOrigin-RevId: 561049410
2023-08-29 09:17:05 -07:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
huangdarwin
350b394596 Compositor: Add tests for 1, 3, and 5 inputs.
With this, Compositor now handles an arbitrary number of inputs!

PiperOrigin-RevId: 558813361
2023-08-22 15:38:05 +01:00
rohks
e0d3cad8bb Add fields next object request (nor) and next range request (nrr)
Added this CMCD-Request fields to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 558317146
2023-08-22 15:25:03 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
huangdarwin
39bc92ffcf Compositor: Use multi-stage rendering and occlude background.
PiperOrigin-RevId: 558110739
2023-08-18 15:28:23 +01:00
huangdarwin
02b347ac66 Test: Reduce tokyo.jpg test file size.
This file decompresses to 3024*4032*4 bytes ~= 48
MB, which leads to OOM errors on some devices.

Use a lower resolution test image for tests, to
still test a portrait image but without hitting
OOM errors.

PiperOrigin-RevId: 557861063
2023-08-18 15:20:34 +01:00
huangdarwin
a3de5978e0 Test: Use unpremultiplied alpha for GL output.
In addition to this being how Alpha should be handled, tests do fail under
API 29 without this CL. Unfortunately while these tests do fail under API 29
without this CL, we currently only run this test on API 33 emulators, so we
didn't catch this failure earlier (until compositor tests on mh failed on all
devices under API 29 and succeeded on all at or over 29).

PiperOrigin-RevId: 557781757
2023-08-18 15:11:12 +01:00
ibaker
eb63ed5871 Add test for Mp3Extractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING_ALWAYS
This test uses an existing CBR test asset, and shows that (as expected)
`isSeekable = true` even in the `unknown_length` dump file.

PiperOrigin-RevId: 557455364
2023-08-16 14:34:21 +01:00
huangdarwin
0466bd7957 Effect: Add alpha effect.
This effect can scale Alpha values.

This is needed for testing Compositor, and coincidentally was requested
by a partner in the past. This also will generally be more useful in
full multi-asset, where we may want to have one input have different alpha
values than another

Also, update fragment_shader_copy_es2.glsl to not throw away alpha values.
This means ThumbnailStripShaderProgram (the only place this glsl is used) will
consider input alpha and output alpha appropriately

PiperOrigin-RevId: 555915092
2023-08-11 20:31:21 +00:00
huangdarwin
fd5784455c Compositor: Add test for correct input timestamps.
To ensure that for each output bitmap from the compostor, the right input
timestamps were used.

Only applied on a subset of tests to avoid needing to upload+maintain
too many files/size in the test binary, especially when it would test
duplicate behavior

PiperOrigin-RevId: 555222530
2023-08-10 12:29:21 +00:00
andrewlewis
bf5c4d8078 Use golden data generated on emulator for tests based on london.jpg
PiperOrigin-RevId: 555148673
2023-08-10 12:22:31 +00:00
Tianyi Feng
ef54364478 Merge pull request #528 from zgzong:patch-2
PiperOrigin-RevId: 554869426
2023-08-10 12:09:43 +00:00
Googler
4fe55b8b63 Return the correct output buffer from audio processing pipeline
PiperOrigin-RevId: 554851456
2023-08-10 12:07:15 +00:00
kimvde
f43146718c Add CompositionPlayer unit test
Test the audio side only because the video side won't be testable with
unit tests once we add the OpenGL components.

PiperOrigin-RevId: 554508703
2023-08-10 11:59:42 +00:00
andrewlewis
691bf93032 Use lower resolution bitmap for tests
The test file decompresses to about 48 MB, which can cause an
`OutOfMemoryError` on lower end devices.

Also, a test appears to apply a rotation to this bitmap without downscaling
first, leading to exceeding a maximum texture width/height of 4096 on some
devices.

Replace the test file with one that has a quarter of the original's
width/height so that these tests can pass reliably even on lower end
devices.

PiperOrigin-RevId: 554507185
2023-08-10 11:58:25 +00:00
tonihei
71591782f0 Adjust periodEnded logic further
Periods that are not the last in the manifest are always ended
and non-dynamic periods are also always ended by definition.

Also, some periods may not be able to declare their final duration
accurately and we need to account for some inaccuracy. This can be
done by testing whether no further full segment within the duration
can be expected.
2023-08-08 17:47:23 +01:00
tonihei
a3c1d5be9d Reset loading period after reading discontinuity that requires loading
This is the same change as 8655429af7, just on the second call site
of readDiscontinuity. The tests didn't cover this case yet because
they never queued more than 2 items in a playlist.

PiperOrigin-RevId: 553485244
2023-08-07 11:30:38 +00:00
tofunmi
dfd668238a Create SingleSampleSeekMap and use in pngExtractor
PiperOrigin-RevId: 553191571
2023-08-07 11:27:27 +00:00
Googler
e87f0d5580 Add a thumbnail strip effect that tiles frames horizontally.
The size of the thumbnail strip and the timestamp of the video frames to use must be specified by the user of the effect.

PiperOrigin-RevId: 552809210
2023-08-07 11:15:10 +00:00