257 Commits

Author SHA1 Message Date
rohks
fe199455e7 Add QuickTime (Classic) support to Mp4Extractor
PiperOrigin-RevId: 566272132
2023-09-18 05:29:29 -07:00
tofunmi
16b0ea850f Support multiple streams in the ImageRenderer
PiperOrigin-RevId: 565410924
2023-09-14 10:37:58 -07:00
claincly
1d8135e563 Add test for MultiInputVideoGraph
This test composites the first frame from two video inputs.

PiperOrigin-RevId: 565090338
2023-09-13 10:28:07 -07:00
ibaker
885ddb167e Add general-purpose overflow-resistant divide+multiply util method
This is equivalent to the existing `scaleLargeTimestamp` method with the
following changes/improvements:
* No longer specific to timestamps (there was nothing inherently
  time-specific about the logic in `scaleLargeTimestamp`, but the name
  and docs suggested it shouldn't be used for non-timestamp use-cases).
* Additional 'perfect division' checks between `value` and `divisor`.
* The caller can now provide a `RoundingMode`.
* Robust against `multiplier == 0`.
* Some extra branches before falling through to (potentially lossy)
  floating-point math, including trying to simplify the fraction with
  greatest common divisor to reduce the chance of overflowing `long`.

This was discussed during review of 6e91f0d4c5

This change also includes some golden test file updates - these
represent a bug fix where floating-point maths had previously resulted
in a timestamp being incorrectly rounded down to the previous
microsecond. These changes are due to the 'some more branches' mentioned
above.

PiperOrigin-RevId: 564760748
2023-09-12 10:22:57 -07:00
tofunmi
23665090ed Support Jpeg image track extraction in exoplayer
PiperOrigin-RevId: 563818198
2023-09-08 12:15:10 -07:00
tofunmi
be5fa6d130 Support image track extraction for JPEG
PiperOrigin-RevId: 563746945
2023-09-08 07:32:10 -07:00
samrobinson
866d62dd34 Improve CompositionExportTest assertions by using dump files.
PiperOrigin-RevId: 563708666
2023-09-08 04:09:21 -07:00
samrobinson
763dddfbd4 Dump with C.TrackType as key, rather than muxer track index.
Modifying dumping to not required "released" to be called.

Track index is an arbitrary value based on the order of addTrack calls.
Samples are dumped by track (rather than as soon as they are written),
so it's preferable to use a value that provides more context.

By using the track type as a key, dump files will be more deterministic
and will have more similarities when branched.

PiperOrigin-RevId: 563700982
2023-09-08 03:25:13 -07:00
tofunmi
535af35511 Add heif extractor
PiperOrigin-RevId: 563340591
2023-09-07 00:31:04 -07:00
samrobinson
cff2816da4 Add silence generation parameterized test case.
PiperOrigin-RevId: 563098931
2023-09-06 07:34:32 -07:00
samrobinson
92814b84a8 Add alternate MP4 asset with PCM audio.
Audio is Mono 44.1kHz.

Created using:
`ffmpeg -i <input> -c:v copy -c:a pcm_s16be <output>`

PiperOrigin-RevId: 563079553
2023-09-06 05:58:27 -07:00
samrobinson
ef45c0fe5d Add parameterized dump tests for single item exports.
PiperOrigin-RevId: 563075771
2023-09-06 05:35:51 -07:00
tofunmi
a1be9d5a60 Add WebpExtractor
Also adds it to the DefaultExtractorsFactory for use throughout exoplayer

PiperOrigin-RevId: 562784760
2023-09-05 08:23:03 -07:00
sheenachhabra
657856dfe5 Fix/Update TODOs in Mp4Muxer tests
PiperOrigin-RevId: 562728777
2023-09-05 04:04:51 -07:00
tofunmi
9b2668c0ca Add BmpExtractor
Takes single sample extraction logic out of the png extractor and puts it in a helper so it can be used for bmp as well.

PiperOrigin-RevId: 562581815
2023-09-04 10:37:28 -07:00
christosts
70ad637e52 Conditionally output captured data in CapturingRenderersFactory
This change makes capturing components in CapturingRenderersFactory
to dump data only they have previously captured any. This is so we can
extend the CapturingRenderersFactory with more renderers that do not
capture data in pre-existing tests and we don't have to change the
golden files.

PiperOrigin-RevId: 561973645
2023-09-01 09:26:33 -07:00
tofunmi
e15c05fc3c Flip input buffer in the ImageRenderer
also improves tests to hash bitmap now that we know the bitmaps are decoded in roboelectic tests,
as well as some minor fixes in the DefaultImageDecoder

PiperOrigin-RevId: 561946073
2023-09-01 07:12:57 -07:00
huangdarwin
57bc215210 Compositor: Implement OverlaySettings and custom in/out size support.
Implement VideoCompositor support of:
* Different input and output sizes
* CompositorSettings, to customize output size based on input texture sizes
* OverlaySettings, to place an input frame in an arbitrary position on
  the output frame.

Also, refactor Overlay's matrix logic to make it more reusable between
Compositor and Overlays

PiperOrigin-RevId: 561931854
2023-09-01 05:54:15 -07:00
tofunmi
5944adcc78 Add image e2e test
PiperOrigin-RevId: 561887238
2023-09-01 01:31:45 -07:00
tofunmi
bb214b19f9 Sample metadata in the png extractor
PiperOrigin-RevId: 561049410
2023-08-29 09:17:05 -07:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
huangdarwin
350b394596 Compositor: Add tests for 1, 3, and 5 inputs.
With this, Compositor now handles an arbitrary number of inputs!

PiperOrigin-RevId: 558813361
2023-08-22 15:38:05 +01:00
rohks
e0d3cad8bb Add fields next object request (nor) and next range request (nrr)
Added this CMCD-Request fields to Common Media Client Data (CMCD) logging.

PiperOrigin-RevId: 558317146
2023-08-22 15:25:03 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
huangdarwin
39bc92ffcf Compositor: Use multi-stage rendering and occlude background.
PiperOrigin-RevId: 558110739
2023-08-18 15:28:23 +01:00
huangdarwin
02b347ac66 Test: Reduce tokyo.jpg test file size.
This file decompresses to 3024*4032*4 bytes ~= 48
MB, which leads to OOM errors on some devices.

Use a lower resolution test image for tests, to
still test a portrait image but without hitting
OOM errors.

PiperOrigin-RevId: 557861063
2023-08-18 15:20:34 +01:00
huangdarwin
a3de5978e0 Test: Use unpremultiplied alpha for GL output.
In addition to this being how Alpha should be handled, tests do fail under
API 29 without this CL. Unfortunately while these tests do fail under API 29
without this CL, we currently only run this test on API 33 emulators, so we
didn't catch this failure earlier (until compositor tests on mh failed on all
devices under API 29 and succeeded on all at or over 29).

PiperOrigin-RevId: 557781757
2023-08-18 15:11:12 +01:00
ibaker
eb63ed5871 Add test for Mp3Extractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING_ALWAYS
This test uses an existing CBR test asset, and shows that (as expected)
`isSeekable = true` even in the `unknown_length` dump file.

PiperOrigin-RevId: 557455364
2023-08-16 14:34:21 +01:00
huangdarwin
0466bd7957 Effect: Add alpha effect.
This effect can scale Alpha values.

This is needed for testing Compositor, and coincidentally was requested
by a partner in the past. This also will generally be more useful in
full multi-asset, where we may want to have one input have different alpha
values than another

Also, update fragment_shader_copy_es2.glsl to not throw away alpha values.
This means ThumbnailStripShaderProgram (the only place this glsl is used) will
consider input alpha and output alpha appropriately

PiperOrigin-RevId: 555915092
2023-08-11 20:31:21 +00:00
huangdarwin
fd5784455c Compositor: Add test for correct input timestamps.
To ensure that for each output bitmap from the compostor, the right input
timestamps were used.

Only applied on a subset of tests to avoid needing to upload+maintain
too many files/size in the test binary, especially when it would test
duplicate behavior

PiperOrigin-RevId: 555222530
2023-08-10 12:29:21 +00:00
andrewlewis
bf5c4d8078 Use golden data generated on emulator for tests based on london.jpg
PiperOrigin-RevId: 555148673
2023-08-10 12:22:31 +00:00
Tianyi Feng
ef54364478 Merge pull request #528 from zgzong:patch-2
PiperOrigin-RevId: 554869426
2023-08-10 12:09:43 +00:00
Googler
4fe55b8b63 Return the correct output buffer from audio processing pipeline
PiperOrigin-RevId: 554851456
2023-08-10 12:07:15 +00:00
kimvde
f43146718c Add CompositionPlayer unit test
Test the audio side only because the video side won't be testable with
unit tests once we add the OpenGL components.

PiperOrigin-RevId: 554508703
2023-08-10 11:59:42 +00:00
andrewlewis
691bf93032 Use lower resolution bitmap for tests
The test file decompresses to about 48 MB, which can cause an
`OutOfMemoryError` on lower end devices.

Also, a test appears to apply a rotation to this bitmap without downscaling
first, leading to exceeding a maximum texture width/height of 4096 on some
devices.

Replace the test file with one that has a quarter of the original's
width/height so that these tests can pass reliably even on lower end
devices.

PiperOrigin-RevId: 554507185
2023-08-10 11:58:25 +00:00
tonihei
71591782f0 Adjust periodEnded logic further
Periods that are not the last in the manifest are always ended
and non-dynamic periods are also always ended by definition.

Also, some periods may not be able to declare their final duration
accurately and we need to account for some inaccuracy. This can be
done by testing whether no further full segment within the duration
can be expected.
2023-08-08 17:47:23 +01:00
tonihei
a3c1d5be9d Reset loading period after reading discontinuity that requires loading
This is the same change as 8655429af7, just on the second call site
of readDiscontinuity. The tests didn't cover this case yet because
they never queued more than 2 items in a playlist.

PiperOrigin-RevId: 553485244
2023-08-07 11:30:38 +00:00
tofunmi
dfd668238a Create SingleSampleSeekMap and use in pngExtractor
PiperOrigin-RevId: 553191571
2023-08-07 11:27:27 +00:00
Googler
e87f0d5580 Add a thumbnail strip effect that tiles frames horizontally.
The size of the thumbnail strip and the timestamp of the video frames to use must be specified by the user of the effect.

PiperOrigin-RevId: 552809210
2023-08-07 11:15:10 +00:00
tofunmi
bb699e41c9 Add png extractor
Implementing a basic extractor that reads the whole png file into the trackOutput as one sample.

PiperOrigin-RevId: 551897619
2023-08-01 14:24:47 +01:00
claincly
c221958889 Remove setInputFrameInfo
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).

This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)

- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
  - We bypass the input manager
  - The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
  - The generated frame is sent to texture output and in turn saved to bitmaps
  - We then compare the generated bitmap with the goldens

PiperOrigin-RevId: 551795770
2023-08-01 14:17:59 +01:00
samrobinson
54093a152e Integrate AudioMixer for audio export.
Adds support for compositions with multiple audio sequences.

PiperOrigin-RevId: 550880626
2023-08-01 14:03:22 +01:00
samrobinson
357c458028 Ensure EOS is queued after processing generated silence with effects.
When generating silence for AudioProcessingPipeline, audio never
queued EOS downstream.

Linked to this, when silence followed an item with audio, the silence
was added to SilentAudioGenerator before the mediaItem reconfiguration
occurred. If the silence had effects, the APP would be flushed after
silence queued EOS, resetting APP.isEnded back to false, so AudioGraph
never ended.

Regression tests reproduce failure without fix, but pass with it.

PiperOrigin-RevId: 550853714
2023-08-01 14:02:09 +01:00
jbibik
f4ad26451f Deprecate MimeTypes.TEXT_EXOPLAYER_CUES for APPLICATION_MEDIA3_CUES
Instead of `text/x-exoplayer-cues`, we will use `application/x-media3-cues`.

The prefix should be `application` not `text` since the encoded form is arbitrary bytes, not necessarily text. The name should not reference `exoplayer`, since the Media3 Extractors (which are not part of `exoplayer`) produce this format.

PiperOrigin-RevId: 550181852
2023-08-01 13:50:50 +01:00
tofunmi
7eee15ecb4 Update OverlaySettings to explicitly state overlay transformations
Alters the OverlayShaderProgram implementation to support rotations, however we need to apply the transformations separately in order for them to work as expected so the matrix is removed from the interface in favour of explicit methods.

Adds a rotation test to ensure this ability doesn't regress

PiperOrigin-RevId: 549890847
2023-08-01 13:45:22 +01:00
jbibik
e11e484bc3 Do not emit empty cues in SsaParser
It is not part of the `SubtitleParser` interface's promise to give events for all changes to do with `Cue` objects. So while in the past, we would have `endTimeUs` of one Cue event being the `startTimeUs` of the next one, now we have that encoded in `durationUs` and can skip event with empty `Cues`.

PiperOrigin-RevId: 549629157
2023-08-01 13:41:11 +01:00
ibaker
1cfaae3450 Enable WebVTT subtitles in MkvPlaybackTest
This is no longer flaky because WebVTT subtitles are decoded on the
loading thread (enabled in this test with
`defaultExractorsFactory.setTextTrackTranscodingEnabled(true)` - and
supported for WebVTT since f0f24aa0d4).

PiperOrigin-RevId: 549594291
2023-08-01 13:35:11 +01:00
jbibik
f4c66d82dd TtmlDecoderTest refactor not to use TtmlNode or TtmlStyle
PiperOrigin-RevId: 549584918
2023-08-01 13:30:39 +01:00
ibaker
5d453fcf37 Add an Extractor to parse subtitles before SampleQueue
The end-to-end test output for the overlapping SRT and SSA subtitles
is currently incorrect. They will be fixed in a future change that
updates `TextRenderer` to support this overlap.

The 'extra' samples visible in the extractor test output files are
'empty cue list' samples produced by `SsaParser`. They will go away
when this implementation is updated to remove this behaviour and rely
on `CuesWithTiming.durationUs` instead (the 'empty list' behaviour is
not required by the `SubtitleParser` interface).

PiperOrigin-RevId: 549264593
2023-07-20 10:30:16 +01:00