146 Commits

Author SHA1 Message Date
sheenachhabra
87cd90eb15 Fix TODO in Boxes.java
The fix is as per MP4 spec ISO/IEC 14496-12

PiperOrigin-RevId: 518240043
2023-03-21 14:22:08 +00:00
sheenachhabra
669008f437 Use FakeExtractorOutput in Mp4MuxerMetadataTest
The FakeExtractorOutput dumps data in more readable format
so using that where ever possible.

The MdtaMetadataEntry which contains key and value dumped only "key".
Added fix to dump "value" as well.

PiperOrigin-RevId: 517968996
2023-03-21 14:19:12 +00:00
sheenachhabra
ccdbd3b0a3 Add tkhd, mvhd and mdhd box test cases
PiperOrigin-RevId: 517933485
2023-03-21 14:17:47 +00:00
huangdarwin
e54c7a7c79 Test: Trim 4k60 test file to 300ms.
This was trimmed using the Transformer Demo, from ~4 seconds to 300ms

Otherwise, this test file will be quite large, and folks who clone the repo will
have to clone this content as well. Trimming the file also allows the test to
take less time, while still testing the core purpose of the file, to confirm that 4k extraction/decode/GL/encode/muxing works.

PiperOrigin-RevId: 517408463
2023-03-21 14:15:35 +00:00
huangdarwin
a6479c3d10 Test: Move 4k60 test file to local from remote.
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.

export4k60 test timing on Pixel 7:

|Condition                 |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL)      |4253         |14                    |
|Remote file, googleguest  |5510         |17                    |
|Remote file, 4G           |11423        |25                    |

PiperOrigin-RevId: 516815462
2023-03-16 15:31:56 +00:00
sheenachhabra
edfef0a879 Add test for Mp4Muxer robustness
With the Mp4Muxer, while writing a large file if the something unexpected happens and muxer is not closed properly then it still outputs a valid MP4 file with partial data.

PiperOrigin-RevId: 516572804
2023-03-16 15:31:03 +00:00
sheenachhabra
40dfe9f07d Add muxer end to end test for av1 video format
PiperOrigin-RevId: 516499906
2023-03-14 15:55:44 +00:00
sheenachhabra
7b8c562d7b Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 07:56:52 +00:00
sheenachhabra
7031d2c6f4 Remove sample_h265.mp4 test file
The h265 encoding is getting covered in HDR test case (hdr10-720p.mp4) so having a separate asset is redundant here.

PiperOrigin-RevId: 515347662
2023-03-14 07:49:28 +00:00
sheenachhabra
2f01f9c53b Add support for reading location data in MP4 extractor
The geodata is stored in the "udta" box as per MediaMuxer reference
https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/MPEG4Writer.cpp;drc=master;l=5588

PiperOrigin-RevId: 515095127
2023-03-14 07:46:28 +00:00
samrobinson
3264cb8271 Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 11:56:53 +00:00
kimvde
0f8b67b875 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 11:52:52 +00:00
sheenachhabra
a81a5f2699 Create orientation matrix byte array using util method
PiperOrigin-RevId: 514375835
2023-03-07 11:51:13 +00:00
sheenachhabra
f23d6c1dc1 Fix nit issues in muxer end to end tests
Renamed MuxerEndToEndTest.java to Mp4MuxerEndToEndTest.java to align it with class under test.
Removed muxed prefix from dump file name because Mp4 implicitely means muxed only.

PiperOrigin-RevId: 513574681
2023-03-07 11:44:26 +00:00
tofunmi
8ec1f86967 README.md line reflow.
PiperOrigin-RevId: 513289716
2023-03-02 13:23:30 +00:00
tofunmi
e425b6e082 Image frame processing color transfer fix
Used an actual captured image with set color profile for test to minimise the chance of the test flaking. Also renamed the media/bitmap/overlay folder to media/bitmap/input_images for clarity.

PiperOrigin-RevId: 513273353
2023-03-02 13:22:27 +00:00
sheenachhabra
9f51182bb8 Move Mp4MuxerMetadataTest into muxer module
Changes include:
1. Move the test fine into muxer module.
2. Use dump file infra for test cases.
3. Add one additional test for adding float metadata.
4. Few improvements in the code.

In next CL will remove Mp4 term from the file name as we are not using this term in test file names.

PiperOrigin-RevId: 513222506
2023-03-01 17:27:58 +00:00
sheenachhabra
36aa29809d Add Muxer end to end tests
Following test cases are added:
1. Mux H264 video
2. Mux H265 video
3. Mux HDR video

Each test case performs following actions:
1. Extract track and samples from input Mp4 using MediaExtractor.
2. Feed those samples into Mp4 muxer.
3. Use extractor to extract the samples from muxed file and create dump file.

PiperOrigin-RevId: 512589069
2023-02-27 18:50:56 +00:00
huangdarwin
cc1ca860b1 Test: Add format codec info and update HDR10 file to 720p.
Add format codec info, which can make test skipping checks more similar to the
actual Transformer decoder checks.

Also for the test file, the actual format was 720p, but somehow the file name and
media metadata indicated 1080p. This format mismatch led to some decoding errors,
so fix the format (and associated errors). This also allows us to remove the
exception catch in ForceInterpretHdrVideoAsSdrTest, which was included due to
errors from the incorrect format.

PiperOrigin-RevId: 511809507
2023-02-27 18:32:55 +00:00
kimvde
5c54a7dffb Add unit tests for audio track dis(appearing) during export
PiperOrigin-RevId: 511764841
2023-02-27 18:28:14 +00:00
sheenachhabra
2515e39fa2 Move FtypAdjusterTest.java into muxer module
Changes include:
1. Move the test file into muxer module
2. Code improvement
2. Remove dependency from Mp4Slicer library

PiperOrigin-RevId: 511191076
2023-02-27 18:20:48 +00:00
sheenachhabra
85e49fc4e2 Add missing test cases for sample table boxes
Test cases for following boxes are added:
1. stsc
2. co64
3. stss

PiperOrigin-RevId: 511002841
2023-02-27 18:12:57 +00:00
sheenachhabra
a33b0d64c1 Move SampleTableBoxesTest.java into third_party/.../muxer/...
Following changes are included:
1. Move test file into third_party/.../muxer/...
2. Rewrite tests for stts box and stsz box to use dump file infra instead of Mp4Slicer.
3. Remove existing test cases related to stts box which are kind of duplicate as it tries to cover the scenarios which are already covered by "DurationsVuForStts" related test cases.

PiperOrigin-RevId: 510977119
2023-02-27 18:10:23 +00:00
sheenachhabra
f6155557eb Change DumpableBox to dump recursively
PiperOrigin-RevId: 510433206
2023-02-27 17:56:22 +00:00
sheenachhabra
6711a59564 Move BoxesTest.java into muxer module
Following changes are included:
1. Move BoxesTest.java into muxer module.
2. Change test code to use dump file infra.

BoxesTest.java does not cover all the boxes implemented in Boxes.java so created a WI to track it b/269471752

PiperOrigin-RevId: 510219645
2023-02-17 11:54:46 +00:00
huangdarwin
3e2c6797bb Effect: Implement brightness convenience class.
It can be confusing how to use RgbMatrix for app devs not comfortable with image
manipulation, so add a helper class that simply translates rgb values the same
constant brightness value, to update the brightness values.

PiperOrigin-RevId: 509287229
2023-02-13 19:34:34 +00:00
huangdarwin
38dd1489ba Test: Update emulator version from 31 to 33.
For all end-to-end tests, we currently run on a generic API 26 and newer API 31 emulator. Due to some issues on API 31, update to use an API 33 emulator.

Also, update documentation to mention issues on different emulator versions.

PiperOrigin-RevId: 509242743
2023-02-13 17:01:44 +00:00
andrewlewis
5c30165c28 Update overlay test bitmap based on emulator output
PiperOrigin-RevId: 509210836
2023-02-13 14:35:38 +00:00
sheenachhabra
31d04748ef Improve Audio box test and video box test
Following changes are included:
1. Added missing color information in HDR test case.
2. Corrected few namings in AudioBoxesTest.java.
3. Updated corresponding dump files.

PiperOrigin-RevId: 508745206
2023-02-10 21:46:12 +00:00
tofunmi
574424f626 Update frame & texture processors to handle SDR image input
GLEffectsFrameProcessor, MatrixShaderProgram and FinalMatrixShaderProgramWrapper are currently setup to handle the input frames coming from an external input (i.e. a video decoder). Image input is loaded into Bitmap objects at the start of the pipeline, so they are not produced externally. The changes provide a way for the frame processing pipeline to handle this "internal" (i.e. non-external) input.

PiperOrigin-RevId: 508645244
2023-02-10 14:32:07 +00:00
sheenachhabra
ae17da6e97 Move VideoBoxesTest.java
The existing implementation depends on an internal library (Mp4Slicer) to assert the output. We have removed the dependency on internal library and used golden data to compare the output.

PiperOrigin-RevId: 508401527
2023-02-09 17:52:56 +00:00
sheenachhabra
a39b9d9175 Move AudioBoxesTest.java to third_party/.../muxer/...
PiperOrigin-RevId: 508062795
2023-02-14 14:21:09 +00:00
michaelkatz
4854e771d7 Encapsulate Opus frames in Ogg during audio offload
PiperOrigin-RevId: 508053559
2023-02-08 14:12:06 +00:00
kimvde
16db2bd0a1 Add an API entry point to pass a Composition
PiperOrigin-RevId: 508031337
2023-02-08 14:11:13 +00:00
tofunmi
eb6c1a5254 Add sRGB eotf to overlay fragment shader.
The eotf is needed so that overlay (image) colors are correctly interpreted and mixed the linear video colors.

Also replaces the 100winners.png with "homemade" image file.

Added GlEffectsFrameProcessor test to justify that the color looks correct at the end of frame processing.

PiperOrigin-RevId: 506290309
2023-02-02 15:32:08 +00:00
christosts
c6569a36fb Merge pull request #10793 from fraunhoferfokus:dash-thumbnail-support
PiperOrigin-RevId: 506261584
2023-02-02 15:30:08 +00:00
tonihei
00436a04a4 Fix timestamp comparison for seeks in fMP4
When seeking in fMP4, we try to extract as little samples as possible
by only starting at the preceding sync frame. This comparison should
use <= to allow sync frames at exactly the seek position.

Issue: google/ExoPlayer#10941

#minor-release

PiperOrigin-RevId: 505098172
2023-02-01 10:26:52 +00:00
samrobinson
dfa98ae791 Generate silent audio if no audio track is available.
To always generate silent audio, `removeAudio(true)` can be used in conjunction.

PiperOrigin-RevId: 502814315
2023-01-18 12:03:59 +00:00
huangdarwin
7878615f22 Effect: Automatically save bitmaps in pixel test.
Also, omit the "actual" label from output files, as this boilerplate isn't necessary
(it doesn't disambiguate between any other saved filename like "expected").

PiperOrigin-RevId: 502378188
2023-01-17 02:10:24 +00:00
huangdarwin
d1e03a41ab HDR: Implement PQ to SDR tone-mapping.
Tested manually on the Pixel 7 and Samsung S10.

PiperOrigin-RevId: 501626354
2023-01-17 01:59:21 +00:00
kimvde
63b8cae263 Add a queue at the start of the buffer SamplePipelines
This improves performance and makes the code more intuitive.

PiperOrigin-RevId: 501220234
2023-01-17 01:48:51 +00:00
kimvde
98bc817fe7 Fix sample interleaving
Whether to write a sample or not was based on the timestamp of the
previous sample, rather than the current sample.

PiperOrigin-RevId: 500195279
2023-01-10 18:35:11 +00:00
tofunmi
18f58d1790 Add TextureOverlay.configure() to configure overlay before frame processing
This change allows an overlay to be sized with respect to the video dimensions.

PiperOrigin-RevId: 497337734
2023-01-04 21:43:33 +00:00
tofunmi
65dcfbd727 Add support for customizing overlay anchor point.
PiperOrigin-RevId: 496956035
2023-01-04 21:32:10 +00:00
huangdarwin
b8f88a8bae HDR: Use local instead of remote file for testing.
This should allow us to focus on HDR failures instead of network buffering
failures when debugging HDR issues.

These files are each used on several files, so it should be more worth the
test binary impact to move these files to local first.

Locally, tests did take less time after this diff

PiperOrigin-RevId: 496398130
2022-12-21 15:20:29 +00:00
tofunmi
3708a4f13e Create dynamically created shaders for multiple overlays.
PiperOrigin-RevId: 496379904
2022-12-21 15:18:26 +00:00
huangdarwin
a66dae8bfa HDR: Implement GlEffectsFrameProcessor HLG to SDR tone-mapping.
Adds COLOR_TRANSFER_GAMMA_2_2, to match behavior in other HDR to SDR tone mapping
implementations.

PiperOrigin-RevId: 495371736
2022-12-15 16:11:42 +00:00
andrewlewis
beee4732fb Generate complete silent audio frames
`SilentAudioGenerator` could output a fractional audio frame, and this could cause downstream components to throw because of trying to read a complete audio frame but only seeing a partial one.

Calculate the output buffer size based on the frame size (which is a no-op for stereo 16-bit audio) and calculate a total number of frames to output then multiple by the frame size.

PiperOrigin-RevId: 494992941
2022-12-15 15:58:50 +00:00
tofunmi
feb3b0b919 Add support for changing translucency of overlays through alpha values
Implements milestone 1.3 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493290147
2022-12-12 11:22:33 +00:00
tofunmi
e6cb502bc6 Implement text and drawable overlays
Implements milestone 1.2 of the [overlays implementation plan](https://docs.google.com/document/d/1EcP2GN8k8N74hHZyD0KTqm9oQo5-W1dZMqIVyqVGtlo/edit#bookmark=id.76uzcie1dg9d)

PiperOrigin-RevId: 493282868
2022-12-12 11:21:27 +00:00