542 Commits

Author SHA1 Message Date
claincly
fd376bea07 Add stress test for image transform
PiperOrigin-RevId: 642213253
2024-06-11 03:59:32 -07:00
tofunmi
0d2fdcd025 Add Transformer HDR-only assets sequence effect tests.
PiperOrigin-RevId: 641828107
2024-06-10 02:11:09 -07:00
andrewlewis
3b6175daa7 Fix test names
PiperOrigin-RevId: 641249128
2024-06-07 07:56:30 -07:00
claincly
ea7cfeccb8 Fix timestamps in tests
38a7229d96 changed only some of the timestamps, in fact all
of the timestamps should have the offset removed.

PiperOrigin-RevId: 641226102
2024-06-07 06:04:31 -07:00
claincly
39e572ad6d Fix a concurrency issue that shader capacity is set off GL thread
Also add test to cover transitions between BT709 and 601.

PiperOrigin-RevId: 641224971
2024-06-07 05:59:20 -07:00
claincly
38a7229d96 Unify timestamp handling
Before this change, the timestamps output from composition playback is offset
with the renderer offset. After this change, the offset is removed and the
timestamp behaviour converges with Transformer, that is, the timestamps of
video/images frames will follow that of the composition. For example, with a
composition of two 10-s items, clipping the first with 2s at the start, the
timestamp of the first frame in the second item, will be 8s.

PiperOrigin-RevId: 641121358
2024-06-06 21:36:15 -07:00
sheenachhabra
ec0af5a7e0 Add CodecDetails to ExportException
This will replace the existing free-form string in the error message

PiperOrigin-RevId: 640954158
2024-06-06 11:01:29 -07:00
dancho
9c4a532d5c Add LanczosResample effect to scale videos in Transformer
Add SeparableConvolution.configure(inputSize) to allow effect configuration
depending on input dimensions.
Add LanczosResample.scaleToFit method to scale input images to fit inside
given dimensions.

PiperOrigin-RevId: 640498008
2024-06-05 06:00:25 -07:00
dancho
5371ad406b Skip AV1 test on devices that don't support AV1 HDR encoding
Skip AV1 HDR test on some devices.

PiperOrigin-RevId: 640135455
2024-06-04 06:44:23 -07:00
dancho
2bb12de00d Drop API requirement from SeparableConvolution
Switch from 4-channel RGBA_16F lookup texture to 1-channel R_16F.
Do not use a bitmap when creating the lookup table texture.
Instead, fill the texture directly.
Do not manually convert 32-bit float to 16-bit. Instead, let OpenGL
libraries do this for us.

PiperOrigin-RevId: 639717235
2024-06-03 03:31:41 -07:00
kimvde
151a2be141 Regenerate TransformerHdrTest goldens after removing degammaing
Degammaing has been removed in cb4b2ea55c. The goldens for
TransformerHdrTest (previously TransformerSequenceEffectTestWithHdr)
were not regenerated because the test wasn't running due to its name
(fixed in e41a966237).

PiperOrigin-RevId: 638645635
2024-05-30 07:32:22 -07:00
kimvde
e41a966237 Rename test so that it runs on MH
PiperOrigin-RevId: 638237657
2024-05-29 04:35:21 -07:00
samrobinson
21eb482baf Add an experimental analyzer mode to Transformer.
PiperOrigin-RevId: 637926059
2024-05-28 09:31:40 -07:00
dancho
3c998ac408 Add a flag to control whether input bitmap resampling can be skipped
Add DefaultVideosFrameProcessor experimental flag that controls
whether input Bitmaps are sampled once for a repeating sequence of
output frames with the same contents, or once for each output frame.

PiperOrigin-RevId: 637921350
2024-05-28 09:15:23 -07:00
dancho
02df88e5d9 Speed up image to video Export
Only sample from input bitmap when the input image has changed.
Introduce GainmapShaderProgram.newImmutableBitmap API that signals
input bitmap changes to GainmapShaderProgram (DefaultShaderProgram).

PiperOrigin-RevId: 637920207
2024-05-28 09:12:24 -07:00
tofunmi
9622411b50 Add support for ultra HDR overlays
PiperOrigin-RevId: 637863706
2024-05-28 05:28:03 -07:00
tofunmi
db6144e7dd Maintain a consistent luminance range across HDR content in effects
PQ and HLG have different luminance ranges (max 10k nits and max 1k nits resp). In GL, colors work in a normalised 0 to 1 scale, so for PQ content, 1=10k nits and and for HLG content, 1=1k nits.

This cl scales and normalises PQ content appropriately so that all HDR content works in the HLG luminance range. This fixes two things

1. Conversions between HLG and PQ are "fixed" (before the output colors looked too bright or too dark depending on which way you are converting)
2. color-altering effects will be able to work consistently across HLG and PQ content

1 is tested in this cl. 2 will be tested when ultra HDR overlays are implemented, both cases have been manually tested to ensure the output looks correct on a screen.

PiperOrigin-RevId: 636851701
2024-05-24 03:05:34 -07:00
samrobinson
414dbebc2b Assert file size in E2E android tests that claim to complete.
Remove redundant test logic to add file size to ExportResult because
the file size is already added to export result as part of an export
finishing.

PiperOrigin-RevId: 636499236
2024-05-23 04:17:19 -07:00
dancho
c2fb2f1520 Work around 1080p export failures on certain devices
Fall back to using software decoder for 1920x1080 for certain
devices.

PiperOrigin-RevId: 636132298
2024-05-22 05:38:50 -07:00
samrobinson
8e4d82b012 Assert file exists before trying to re-decode for test assertions.
PiperOrigin-RevId: 635748820
2024-05-21 03:41:48 -07:00
dancho
ae240606db Move bitmap coordinate flip out of fragment shader
Fragment shaders in OpenGL ES shader language aren't guaranteed
to support highp, required to correctly represent pixel coordinates
inside large images (e.g. 1920x1080).
This change moves coordinate mirroring for images out of fragment shader.

Fixes http://Issue: androidx/media#1331

PiperOrigin-RevId: 635732208
2024-05-21 02:31:43 -07:00
dancho
b047e81e02 Work around SurfaceTexture implicit scale
If MediaCodec allocates passes an image buffer with a cropped region,
SurfaceTexture.getTransformMatrix will cut off 2 pixels from each dimensions.
The resulting videos will appear a little stretched.

This patch inspects the SurfaceTexture transform matrix, and guesses what the
unscaled transform matrix should be.
Behind experimentalAdjustSurfaceTextureTransformationMatrix flag

PiperOrigin-RevId: 635721267
2024-05-21 01:46:21 -07:00
dancho
b9ec24a269 Fix AV sync for sequences with audio track shorter than video
For each item, AudioGraphInput now pads the input audio track with silence
to the duration given in onMediaItemChanged.

Possibly resolves Issue: androidx/media#921 .

PiperOrigin-RevId: 634753721
2024-05-17 06:54:39 -07:00
ibaker
0e5a5e0294 Publish CompositionPlayer for playing compositions
This class is not ready for production app usage yet, so it is still
marked `@RestrictTo(LIBRARY_GROUP)` for now. Apps can experiment with it
in a non-prod context by suppressing the associated lint error.

* Issue: androidx/media#1014
* Issue: androidx/media#1185
* Issue: androidx/media#816

PiperOrigin-RevId: 633921353
2024-05-15 05:55:08 -07:00
sheenachhabra
3a3145521b Replace Transformer.Muxer interface with Muxer.Muxer
PiperOrigin-RevId: 633193701
2024-05-13 06:54:19 -07:00
sheenachhabra
5950e884f6 Skip TransformerPauseResumeTest on vivo 1820 and vivo 1906
The process crashes unexpectedly on these devices.

The new changes skips running these tests instead of marking them pass, when
the device needs to be skipped.

PiperOrigin-RevId: 633183638
2024-05-13 06:07:20 -07:00
sheenachhabra
71e36ac6d2 Remove missing frame workaround from TransformerPauseResumeTest
Two devices were producing B-frames earlier and were causing
frame mismatch. So we had added a workaround for them.
Those devices does not produce B-frames now
after disabling high profile on them, so we don't need workaround now.

PiperOrigin-RevId: 633127755
2024-05-13 01:35:34 -07:00
tofunmi
1749726bc9 Texture input tests: use separate textures to queue to transformer
Use different textures in calls to queueInputTexture(). Allows the texture to be deleted one it is used in transformer and effect.

PiperOrigin-RevId: 632430866
2024-05-10 02:41:51 -07:00
tofunmi
753f607a81 Test: Remove the invalid dataspace device list when asserting sdr
PiperOrigin-RevId: 632137474
2024-05-09 07:07:51 -07:00
Luyuan Chen
766ff44a2c Fix review comment 2024-05-08 00:11:21 +00:00
Patrik Aradi
52adaf8d26 implement top down approach for passing input source id 2024-05-08 00:07:38 +00:00
Patrik Aradi
177f1f33d0 Fix indeterminate z-order of EditedMediaItemSequences by passing sequenceIndex when registeringInput 2024-05-08 00:07:37 +00:00
claincly
bef3d518d2 Revise seeking in ExternalTextureManager (ETM)
After this CL, DVFP waits for flushing until all frames registered previously
arrives.

Previously, ETM records the difference between the number of registered frames,
and the number of frames arrivd on the SurfaceTexture, when flushing. (Note
that ETM is flushed the last in the chain, as flushing is done backwards from
FinalShaderProgramWrapper). ETM then waits until the number of frames arrive
after flush.

The normal flow is, MediaCodecVideoRenderer (MCVR) registers a new decoded
frame, in `processOutputBuffer()` to DVFP, MCVR call `codec.releaseOutputBuffer()`
to have MediaCodec render the frame, and then the frame arrives in DVFP's ETM.

However there might be a discrepancy. When registering the frame, ETM records
the frame on the calling thread, ~instantly. Later when the rendered frame
arrive, ETM records a frame is available on the task executor thread (or
commonly known as the GL thread). More specifically, when a frame arrives
in `onFrameAvailableListener`, ETM posts all subsequent processing to
the task executor. When seeking, the task executor is flushed as the first
step. It might be a frame that has already arrived on ETM, and the processing
of such frame has already been queued into the task executor; only to be
flushed as a result of flushing the task executor. If this happens, the frame
is considered to be never have arrived. This causes a freeze on the app,
because ETM'll wait until this frame arrives to declare flushing has completed.

PiperOrigin-RevId: 631524332
2024-05-07 13:15:46 -07:00
tofunmi
b2d30a5722 Fallback to hevc transcoding for hdr AV1 files
PiperOrigin-RevId: 631362735
2024-05-07 03:34:31 -07:00
sheenachhabra
24305c043e Remove forCancellation flag from Transformer/Muxer.release() method
PiperOrigin-RevId: 630337930
2024-05-03 03:18:24 -07:00
claincly
d59f1cb6e3 Add test for seeking composition playback
PiperOrigin-RevId: 630184355
2024-05-02 14:46:39 -07:00
tofunmi
cb4b2ea55c remove degammaing: change setSdrWorkingColorSpace default
The second stage of the changes remove the conversion to linear colors in the SDR effects pipeline by default.

also resolves Issue: androidx/media#1050

PiperOrigin-RevId: 630108296
2024-05-02 10:35:48 -07:00
claincly
45ccc6978a Add setVideoSurface override
Also adds first frame rendered test for playing back compositions.
- This test checks the output pixels using an `ImageReader` to retrieve the
output bitmap

PiperOrigin-RevId: 630100817
2024-05-02 10:14:07 -07:00
tofunmi
28655d27a7 test: use Ascii.equalsIgnoreCase for device model matching
PiperOrigin-RevId: 629349367
2024-04-30 02:54:38 -07:00
tofunmi
7089253bef Effect:migrate setEnableColorTransfers to setSdrWorkingColorSpace
Part of a two stage change to remove the conversion to linear colors in the SDR effects pipeline by default. Changes the boolean to an intdef, introducing a third option that gets all sdr input into the same colorspace.

This is a planned API breaking change, but this change should not change the behavior of the pipeline.

PiperOrigin-RevId: 629013747
2024-04-29 03:26:58 -07:00
tofunmi
5ee6c9b16d ultraHdr test: add checks for devices with invalid color formats
PiperOrigin-RevId: 628101730
2024-04-25 09:37:16 -07:00
kimvde
c87940eb27 AudioGraph refactoring: configure mixer and add sources in one place
PiperOrigin-RevId: 628076773
2024-04-25 08:02:00 -07:00
tofunmi
73f614b14d Effect: support disabling color transfers when HDR->SDR tonemapping
also makes the setter more flexible by ignoring the value of the setter when the output is hdr rather than throwing (since all HDR content must be have a linear color space)

PiperOrigin-RevId: 627388436
2024-04-23 08:02:30 -07:00
sheenachhabra
b813bd3976 Use silence audio in RawAssetLoader tests
This is avoid dependency on `MediaExtractor` whose
behaviour is not consistent across different API versions.

PiperOrigin-RevId: 627330485
2024-04-23 03:38:31 -07:00
kimvde
8da6938782 Always initialize VideoSink in renderer
Instead of initializing the video sink outside the renderer with an
empty format for composition preview, we initialize it in the renderer
with the input format for video.

PiperOrigin-RevId: 627313708
2024-04-23 02:20:18 -07:00
sheenachhabra
03a041c452 Pass BufferInfo in writeSampleData() method in Transformer/Muxer.java
This is to eventually replace Transformer/Muxer.java with Muxer/Muxer.java

PiperOrigin-RevId: 627043808
2024-04-22 08:23:34 -07:00
sheenachhabra
86ef571644 Replace trackIndex with TrackToken in Transformer
This is to eventually replace Transformer/Muxer.java with Muxer/Muxer.java

PiperOrigin-RevId: 627027254
2024-04-22 07:20:38 -07:00
tofunmi
8ae9e81196 add test for speed adjustments, audio removed and a forced audio track
add coverage for key use case for speed adjustments

PiperOrigin-RevId: 626957912
2024-04-22 02:00:46 -07:00
tofunmi
fb037b9847 TimestampWrapper: fix signaling input capacity
fixes https://github.com/androidx/media/issues/821

PiperOrigin-RevId: 626407880
2024-04-19 10:40:59 -07:00
sheenachhabra
4aa2fb883f Add RawAssetLoader and deprecate TextureAssetLoader
All the changes are based on existing implementations
and their related tests.

No new functionality is being added.

PiperOrigin-RevId: 625756836
2024-04-17 11:48:43 -07:00