484 Commits

Author SHA1 Message Date
tofunmi
a98b33b03c Replace glFinish in texture output with Gl sync token
Providing the sync token in the api allows the client to decide which waiting method they would like to use depending on the use case, allowing them to optimise if possible.

PiperOrigin-RevId: 543997311
2023-06-29 23:07:56 +00:00
tofunmi
8cecb93570 Texture Input: Add GLSyncToken to the OnInputFrameProcessedListener
PiperOrigin-RevId: 543867944
2023-06-29 23:02:20 +00:00
claincly
f98a10f3f2 Move GlShaderProgram creation away from VFP creation
PiperOrigin-RevId: 543773418
2023-06-29 23:00:11 +00:00
kimvde
b358d075a1 Replace usages of deprecated TransformationRequest setter
PiperOrigin-RevId: 543765376
2023-06-29 22:59:06 +00:00
sheenachhabra
8735382590 Move SSIM related files to test_util
PiperOrigin-RevId: 543444043
2023-06-29 22:51:20 +00:00
tonihei
2322462404 Do not trim audio samples by changing their timestamp
MP4 edit lists sometimes ask to start playback between two samples.
If this happens, we currently change the timestamp of the first
sample to zero to trim it (e.g. to display the first frame for a
slightly shorter period of time). However, we can't do this to audio
samples are they have an inherent duration and trimming them this
way is not possible.

#minor-release

PiperOrigin-RevId: 543420218
2023-06-29 22:50:04 +00:00
claincly
12584fbb20 Add test for force EOS when a decoder drops frames
This is for the timer added in abd96598d9

PiperOrigin-RevId: 542571387
2023-06-23 16:42:46 +00:00
tofunmi
09fe0d7390 Correct the DefaultFrameDroppingShaderProgram logic
copies the previous texture to a temp texture to ensure the correct frame is queued.

#minor-release

PiperOrigin-RevId: 541972349
2023-06-22 15:29:41 +00:00
claincly
4cb5f38007 Increase pixel difference
PiperOrigin-RevId: 541860433
2023-06-20 14:05:07 +01:00
huangdarwin
949e9cbd96 GL: Rename GL methods.
To more accurately describe what they do, especially as Compositor will
starts to use more contexts or threads, and it's important to know what
needs to be reset/recreated/focused before what methods.

PiperOrigin-RevId: 541010135
2023-06-19 16:29:03 +01:00
huangdarwin
daa42322d7 GL: Remove configAttributes from createFocusedPlaceholderEglSurface.
PiperOrigin-RevId: 540901189
2023-06-19 16:27:02 +01:00
claincly
0d3082e6ad Reduce the pixel differnce in TransformerSequenceEffectTest
As the test images are re-gened on the emulators.

PiperOrigin-RevId: 540851288
2023-06-19 16:23:13 +01:00
tonihei
51fb72b00d Replace deprecated NullableType from checkerframework with our own one
The existing NullableType has been deprecated 5 years ago and causes
crashes in Kotlin apps because Kotlin doesn't recognize this annotation
as a nullable type annotation.

While we can't align on a single @Nullable annotation yet, we can at
least replace this one by JSR305's @Nonnull(MAYBE) as it fulfils all
requirements, including full Kotlin compatiblity. To avoid the
cumbersome name, we can redefine it as our own @NullableType
annotation. (We can't use @Nullable to avoid name clashes with the main
@Nullable annotation from AndroidX)

Issue: google/ExoPlayer#6792
PiperOrigin-RevId: 540497469
2023-06-19 16:08:20 +01:00
claincly
5961637c0a Add test for per-MediaItem effect
The test transcodes four media items with distinct effects, keeping one frame
each; extracts the four frames in the produced video, and compares them against
the expected results.

PiperOrigin-RevId: 539697344
2023-06-14 20:27:37 +01:00
claincly
4b1ac2f172 Factor out video decoding and fix two minor issues
1. Not treating 0 as valid buffer index
2. Not handling the case the last frame is a comparison frame

PiperOrigin-RevId: 539607482
2023-06-12 14:09:48 +00:00
tofunmi
219212d54f rename videoFrameProcessorTestRunner.queueInputBitmap
This methods currently registers and queues the bitmaps Will need to have a method that just queues bitmaps in future CLs.

#minor-release

PiperOrigin-RevId: 539605628
2023-06-12 14:06:22 +00:00
huangdarwin
0c924fcb40 Test: Add no-op effect test for GL tone mapping.
To ensure no regressions for the potentially confusing pipeline of:
* HDR electrical -> SDR linear EOTF+OOTF, and
* SDR linear -> SDR electrical OETF

PiperOrigin-RevId: 538741079
2023-06-08 12:46:34 +00:00
tofunmi
ce203ccfed DefaultVideoFrameProcessorMultipleTextureOutputPixelTest fixes & cleanup
PiperOrigin-RevId: 538495675
2023-06-08 12:45:07 +00:00
huangdarwin
9dad207603 Test: Move duplicated GL tone mapping logic into helper methods.
PiperOrigin-RevId: 538491957
2023-06-08 12:43:40 +00:00
huangdarwin
adf53b4d50 Effect: Add multiple texture output test.
Confirms that multiple textures can be output, and that timestamps and pixels
are as expected.

PiperOrigin-RevId: 538459296
2023-06-08 12:37:00 +00:00
andrewlewis
828a05e0fa Log additional information on test runner timeout
PiperOrigin-RevId: 538437142
2023-06-08 12:35:36 +00:00
tofunmi
2af5752785 Revert AndroidTestUtil.canDecode to use EncoderUtil.findCodecForFormat
208eefc0fd introduced using `DefaultDecoderFactory.getDecoderInfo(format) != null` caused certain tests not to be skipped when they were expected to be, creating more mh failures.

PiperOrigin-RevId: 537820370
2023-06-06 17:57:47 +00:00
andrewlewis
74478f2478 Tidy color info checking tests
ExoPlayer extractors (backing `MetadataRetriever`) now parse the color format
from the bitstream so using `MetadataRetriever` should be an equivalent but
more lightweight way to verify the color info.

Also remove try/catch blocks in test code calling into these methods, and add
skipping based on decoder capabilities in the cases where it was missing.

PiperOrigin-RevId: 537789483
2023-06-05 09:51:01 +00:00
huangdarwin
c52130a212 Test: Add special effects for forcing transcode.
Previously, we would apply a general effect to signal wanting to transcode.

PiperOrigin-RevId: 537034455
2023-06-02 09:16:52 +00:00
claincly
208eefc0fd Fix codec's MIME type is not used
In some cases the codec selected for decoding has a different MIME type than
the media. In thoses cases Transformer continued to use the media's MIME type
and that caused codec configuration failures.

Removed `EncoderUtil.findCodecForFormat()` as we stopped using the method it
uses for finding a codec. Plus, the method is only used in the test.

See also `MediaCodecUtil.getALternativeCodecMimeType()`.

PiperOrigin-RevId: 536683663
2023-05-31 17:57:40 +00:00
tofunmi
a19e07c4d2 Move OnInputFrameProcessedListener into it's own file
PiperOrigin-RevId: 535183521
2023-05-25 15:09:36 +00:00
huangdarwin
a6897aedaa Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
2023-05-24 15:42:47 +01:00
ibaker
0f8ddd5d1a Suffix package in test AndroidManifest.xml files with .test
This is blocking the upgrade to AGP 8.0.1. The Android Studio Upgrade
Assistant refuses to run with the error:
> Modules have the same package for their `main` and `androidTest`
> artifacts
>
> The package specifications in `AndroidManifest.xml` files define the
> same package for the `main` and `androidTest` artifacts, in the
> following modules: `effect`, `muxer`, `transformer`,
`test_exoplayer_abr`, `test_exoplayer_performance`, and `container`.

This change also adds the `.test` suffix to `test/AndroidManifest.xml`
files where it's missing, because although it's not mentioned in the
Upgrade Assistant error it does cause problems later.

```
> Incorrect package="androidx.media3.database" found in source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
  Setting the namespace via the package attribute in the source AndroidManifest.xml is no longer supported.
  Recommendation: remove package="androidx.media3.database" from the source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
```

Issue: androidx/media#409
PiperOrigin-RevId: 533460180
2023-05-19 17:18:22 +01:00
huangdarwin
25fa2df2de Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
2023-05-19 17:10:53 +01:00
tofunmi
5fe10d7652 HDR texture asset loading
PiperOrigin-RevId: 532846248
2023-05-18 12:21:38 +00:00
sheenachhabra
410840c9e1 Add default constructor for InAppMuxer.Factory
PiperOrigin-RevId: 532838813
2023-05-18 12:18:43 +00:00
tofunmi
88642587ac Update mixed input test to include images of different aspect ratios
Makes this test a little more thorough.

PiperOrigin-RevId: 532386515
2023-05-17 08:35:36 +01:00
huangdarwin
62afbe87bb Test: Suppress SSIM timeouts on galaxy fold/flip.
This is a failure only in SSIM, so it seems unlikely we'll prioritize this over
other work or bugs soon. Suppress test failures to reduce triage burden.

PiperOrigin-RevId: 532200729
2023-05-16 09:59:36 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
sheenachhabra
867355fdc5 Rollback of 925aa34e13
*** Original commit ***

Rollback of 65d5132f76

*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

***

PiperOrigin-RevId: 531470081
2023-05-12 13:20:31 +00:00
huangdarwin
5313763eb3 Test: Filter test more precisely, with model instead of manufacturer
PiperOrigin-RevId: 531233480
2023-05-12 10:10:15 +01:00
huangdarwin
674885cfa5 Test: Suppress failing 8k24 SSIM bug.
PiperOrigin-RevId: 531193242
2023-05-12 10:07:55 +01:00
kimvde
e8072ca2c7 Move VideoFrameProcessorFactory setter to Transformer
PiperOrigin-RevId: 531123743
2023-05-11 10:06:21 +00:00
Googler
925aa34e13 Rollback of 65d5132f76
*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

PiperOrigin-RevId: 531056436
2023-05-11 10:03:35 +00:00
tofunmi
7fea435b7e Transformer: Support Texture asset loading
PiperOrigin-RevId: 530888319
2023-05-11 10:01:00 +00:00
sheenachhabra
65d5132f76 Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

PiperOrigin-RevId: 530684007
2023-05-11 09:56:46 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00
huangdarwin
2db2de5993 Transformer: Add codec support for Dolby Vision HDR video
Allow use of H265/H264 codecs for Dolby Vision video.

Also, reflow ExoPlayer code to use this new utility class

PiperOrigin-RevId: 530619388
2023-05-11 09:47:52 +00:00
claincly
213740c456 Enable switching input types via InputMultiplexer.
PiperOrigin-RevId: 529624205
2023-05-05 16:43:58 +00:00
tofunmi
3011a29940 Update InternetConnection check to skip check when uri scheme is null
When we created androidTests, in the past, they always had a URI pointing to a resource, therefore we always had a URI scheme. With texture input, this will not longer be the case (EditedMediaItems's may have URI.EMPTY, which have a null scheme) so we need to check for this so tests don't falsely fail.

PiperOrigin-RevId: 528848411
2023-05-03 17:12:29 +01:00
tofunmi
0902fad55d Effect: split ending frame Processing and getting the output bitmap in tests
PiperOrigin-RevId: 527554408
2023-05-03 16:54:53 +01:00
huangdarwin
3db4205ac6 Test: Add @Ignore to Analysis tests
In ASwB, all Transformer tests can be run by right-clicking on the project and
clicking "Run all tests". We cannot, however, select only some specific test
files within a project (ex. all non-analysis tests) to run tests on.

Add @Ignore to analysis tests, which are not intended to be run anyways
when determining whether Transformer is working on some device. These
tests also don't have proper skipping logic when a device doesn't support a
format, so they can't be run effectively on low-end devices anyways.

This eases manual testing, for example when debugging whether tests all pass
on a device.

When analysis tests are desired to be run, it should be easy to comment out
the @Ignore.

PiperOrigin-RevId: 527289600
2023-04-27 12:25:42 +01:00
huangdarwin
97272c139c Effects: Output to texture without surface in VFP.
Allow the VideoFrameProcessor to output to a texture without an output surface.

Tested by updating texture output tests to no longer output to a surface.

PiperOrigin-RevId: 527244605
2023-04-26 15:51:06 +01:00
huangdarwin
324115f6cf Effects: Allow BT2020 colors with SDR transfers
Previously, I assumed that SDR contents must be BT709, and HDR contents must be
BT2020. Turns out BT2020 is just wide-gamut, and SDR contents / transfers may be
represented in BT2020 color spaces.

Relax the check, so that we don't throw when valid BT2020 SMPTE 170M contents
are input into effects.

PiperOrigin-RevId: 526668347
2023-04-26 15:41:33 +01:00
huangdarwin
de32536e20 HDR: Remove exportUnexpectedColorInfo test.
PiperOrigin-RevId: 526086973
2023-04-24 11:39:09 +01:00