499 Commits

Author SHA1 Message Date
andrewlewis
0b96f4372f Rollback of 95260e28a5
PiperOrigin-RevId: 649003095
2024-07-03 02:52:00 -07:00
andrewlewis
3b7d59ef4f Rollback of 95260e28a5
PiperOrigin-RevId: 648708459
2024-07-02 07:16:27 -07:00
andrewlewis
95260e28a5 Add input type for automatic frame registration
Deprecate `setInputDefaultBufferSize` and `setRequireRegisteringAllInputFrames`
as the new input stream type replaces these (as far as we know they are always
used together).

This is in preparation for supporting asset loaders signaling that they require
these features, specifically for recording from a surface.

PiperOrigin-RevId: 648686087
2024-07-02 05:41:23 -07:00
dancho
476ec607f2 Implement isNoOp for LanczosResample
This allows for transmuxing whenever resolutions match,
and LanczosResample is in the video effects chain.

PiperOrigin-RevId: 645433104
2024-06-21 10:37:19 -07:00
tofunmi
174c49313c support hdr text overlays
adds luminance multiplier to allow the luminance (i.e. brightness) over a text overlay to be scaled

PiperOrigin-RevId: 643047928
2024-06-13 10:41:09 -07:00
claincly
206d2ce8b8 Effect: Delay end of stream until all frames are rendered.
Previously, if renderFramesAutomatically = false, DefaultVideoFrameProcessor
may call onInputStreamProcessedListener after all frames have been
onOutputFrameAvailableForRendering, but before they had all been rendered and
freed.

Delay onInputStreamProcessedListener being called and subsequent DVFP
reconfiguration of effects, until all frames are rendered.

Tested using exoplayer setVideoEffects demo with playlist

PiperOrigin-RevId: 642963100
2024-06-13 06:22:03 -07:00
tofunmi
20d56341fc Remove useHdr from overlaySettings
useHdr is unused option and doesn't make sense is the dynamic range of the overlay and video must match

Also reorders and adds javadoc in line with coding conventions

PiperOrigin-RevId: 642555396
2024-06-12 02:52:36 -07:00
dancho
abf601feaa Support upscaling with LanczosResample
Avoid scaling the lanczos windown function when scaling up.
When scaling up (output > input), we shouldn't scale the window
function or we risk not sampling any of the input pixels.

PiperOrigin-RevId: 641838149
2024-06-10 03:01:11 -07:00
claincly
b42b63f3a3 Change externalShaderProgramInputCapacity to int
The field is only accessed on the GL thread.

PiperOrigin-RevId: 641273595
2024-06-07 09:34:45 -07:00
claincly
39e572ad6d Fix a concurrency issue that shader capacity is set off GL thread
Also add test to cover transitions between BT709 and 601.

PiperOrigin-RevId: 641224971
2024-06-07 05:59:20 -07:00
dancho
9c4a532d5c Add LanczosResample effect to scale videos in Transformer
Add SeparableConvolution.configure(inputSize) to allow effect configuration
depending on input dimensions.
Add LanczosResample.scaleToFit method to scale input images to fit inside
given dimensions.

PiperOrigin-RevId: 640498008
2024-06-05 06:00:25 -07:00
dancho
73dd743929 Support Separable Convolutions with negative coefficients
fragment_shader_separable_convolution_es2.glsl had optimizations that assumed
all convolution coefficients are positive. Support negative coefficients,
and add tests.

PiperOrigin-RevId: 640104741
2024-06-04 04:26:26 -07:00
dancho
2bb12de00d Drop API requirement from SeparableConvolution
Switch from 4-channel RGBA_16F lookup texture to 1-channel R_16F.
Do not use a bitmap when creating the lookup table texture.
Instead, fill the texture directly.
Do not manually convert 32-bit float to 16-bit. Instead, let OpenGL
libraries do this for us.

PiperOrigin-RevId: 639717235
2024-06-03 03:31:41 -07:00
tofunmi
1640becddd Use file inserts and string replacements in overlay shaders
PiperOrigin-RevId: 638251955
2024-05-29 05:39:35 -07:00
dancho
3c998ac408 Add a flag to control whether input bitmap resampling can be skipped
Add DefaultVideosFrameProcessor experimental flag that controls
whether input Bitmaps are sampled once for a repeating sequence of
output frames with the same contents, or once for each output frame.

PiperOrigin-RevId: 637921350
2024-05-28 09:15:23 -07:00
dancho
02df88e5d9 Speed up image to video Export
Only sample from input bitmap when the input image has changed.
Introduce GainmapShaderProgram.newImmutableBitmap API that signals
input bitmap changes to GainmapShaderProgram (DefaultShaderProgram).

PiperOrigin-RevId: 637920207
2024-05-28 09:12:24 -07:00
kimvde
0baa4ff621 Reset input capacity when setting external shader program
Before this CL, externalShaderProgramInputCapacity was not reset when
the external shader program was reset (which occurs when  the
InputSwitcher switches to an input with a different ColorInfo). This is
due to a regression introduced in bef3d518d2.

PiperOrigin-RevId: 637869215
2024-05-28 05:55:00 -07:00
tofunmi
9622411b50 Add support for ultra HDR overlays
PiperOrigin-RevId: 637863706
2024-05-28 05:28:03 -07:00
tofunmi
8feaf82768 UltraHdr: Fix calculations to apply gainmap
PiperOrigin-RevId: 636964921
2024-05-24 10:39:36 -07:00
tofunmi
db6144e7dd Maintain a consistent luminance range across HDR content in effects
PQ and HLG have different luminance ranges (max 10k nits and max 1k nits resp). In GL, colors work in a normalised 0 to 1 scale, so for PQ content, 1=10k nits and and for HLG content, 1=1k nits.

This cl scales and normalises PQ content appropriately so that all HDR content works in the HLG luminance range. This fixes two things

1. Conversions between HLG and PQ are "fixed" (before the output colors looked too bright or too dark depending on which way you are converting)
2. color-altering effects will be able to work consistently across HLG and PQ content

1 is tested in this cl. 2 will be tested when ultra HDR overlays are implemented, both cases have been manually tested to ensure the output looks correct on a screen.

PiperOrigin-RevId: 636851701
2024-05-24 03:05:34 -07:00
andrewlewis
174d452bf7 Add missing module registrations
PiperOrigin-RevId: 636506860
2024-05-23 04:50:01 -07:00
samrobinson
b6ce35d741 Migrate debug trace logs to track generic events for Muxer.
Track information is added to the details string where relevant.

PiperOrigin-RevId: 635815866
2024-05-21 08:33:12 -07:00
andrewlewis
2585051294 Move license to top for consistency
PiperOrigin-RevId: 635742699
2024-05-21 03:16:30 -07:00
dancho
ae240606db Move bitmap coordinate flip out of fragment shader
Fragment shaders in OpenGL ES shader language aren't guaranteed
to support highp, required to correctly represent pixel coordinates
inside large images (e.g. 1920x1080).
This change moves coordinate mirroring for images out of fragment shader.

Fixes http://Issue: androidx/media#1331

PiperOrigin-RevId: 635732208
2024-05-21 02:31:43 -07:00
dancho
b047e81e02 Work around SurfaceTexture implicit scale
If MediaCodec allocates passes an image buffer with a cropped region,
SurfaceTexture.getTransformMatrix will cut off 2 pixels from each dimensions.
The resulting videos will appear a little stretched.

This patch inspects the SurfaceTexture transform matrix, and guesses what the
unscaled transform matrix should be.
Behind experimentalAdjustSurfaceTextureTransformationMatrix flag

PiperOrigin-RevId: 635721267
2024-05-21 01:46:21 -07:00
tofunmi
c409623ca0 Create equals method for gainmaps
Gainmaps don't currently have an equals override, only reference equality is checked by Objects.equals(gainmap1, gainmap2). Create an equals method for gainmaps with the fields we care about to ensure we don't incur false positives in our equality checks.

PiperOrigin-RevId: 635510451
2024-05-20 11:06:03 -07:00
samrobinson
c9854cd695 Make flag for debug traces in logcat private and final.
PiperOrigin-RevId: 635465538
2024-05-20 08:38:33 -07:00
samrobinson
07ddc0fca5 Add debug trace for Muxer completely ending.
PiperOrigin-RevId: 635459757
2024-05-20 08:12:57 -07:00
claincly
dc4f20ed84 Fix image seeking
Queue image again after position reset, and reset timestamp iterator.

PiperOrigin-RevId: 635049953
2024-05-18 07:41:14 -07:00
samrobinson
1abcf5c22c Add debug trace log for AssetLoader Renderer format events.
PiperOrigin-RevId: 634474584
2024-05-16 11:32:13 -07:00
samrobinson
5c6f48ecaf Add debug trace logs for AudioMixer & AudioGraph events.
PiperOrigin-RevId: 633973723
2024-05-15 09:06:34 -07:00
ibaker
ed4820cb61 Remove @UnstableApi from package-private files
This annotation is only needed on public classes.

This change also removes the `/* package */` comment from some `public`
classes.

PiperOrigin-RevId: 633864544
2024-05-15 02:26:55 -07:00
samrobinson
e17b3d7f83 Omit "last: []" from debug trace json if less than 10 event logged.
Reduces the boilerplate from the json produced without any information
loss.

PiperOrigin-RevId: 633540269
2024-05-14 04:45:42 -07:00
samrobinson
95c13819af Log DebugTraceUtil events to logcat as they occur.
PiperOrigin-RevId: 633512813
2024-05-14 03:08:43 -07:00
samrobinson
acf1ede644 Add debug trace logs for input/output events for codecs.
PiperOrigin-RevId: 633244952
2024-05-13 10:01:47 -07:00
samrobinson
04ab71a1d4 Remove component names from event strings.
PiperOrigin-RevId: 633212723
2024-05-13 08:06:44 -07:00
samrobinson
30624aae6e Pass component of event to debug trace logEvent as parameter.
PiperOrigin-RevId: 633176557
2024-05-13 05:33:31 -07:00
samrobinson
325f4ec502 Add debug trace JSON nesting per component.
PiperOrigin-RevId: 632424814
2024-05-10 02:13:56 -07:00
samrobinson
fff68f1932 Order DebugTraceUtil JSON output by each component.
PiperOrigin-RevId: 632193924
2024-05-09 10:28:49 -07:00
samrobinson
49fa343127 Remove unused DebugTraceUtil VFP event.
PiperOrigin-RevId: 632109159
2024-05-09 05:00:39 -07:00
claincly
145e0faf3d Make DefaultVideoCompositor use the first registered input as primary
PiperOrigin-RevId: 632079022
2024-05-09 02:21:31 -07:00
claincly
d977eab5f1 Minor code move for readability
PiperOrigin-RevId: 631922318
2024-05-08 14:19:04 -07:00
samrobinson
8b14d71ff0 Handle null extraFormat on entry in DebugTraceUtil.
PiperOrigin-RevId: 631821753
2024-05-08 09:14:06 -07:00
samrobinson
8b78e04ba3 Add time units to DebugTraceUtil logging.
PiperOrigin-RevId: 631764751
2024-05-08 05:29:04 -07:00
Copybara-Service
c5964d197c Merge pull request #1055 from AradiPatrik:z-order-fix
PiperOrigin-RevId: 631616905
2024-05-07 18:45:09 -07:00
Luyuan Chen
766ff44a2c Fix review comment 2024-05-08 00:11:21 +00:00
Luyuan Chen
0403e5881d Format with google-java-format 2024-05-08 00:07:38 +00:00
Patrik Aradi
51a9bcca61 Refactor according to reviews. 2024-05-08 00:07:38 +00:00
Patrik Aradi
52adaf8d26 implement top down approach for passing input source id 2024-05-08 00:07:38 +00:00
Luyuan Chen
3cccecf368 Format with google-java-format 2024-05-08 00:07:38 +00:00