This simplifies ChainingGlTextureProcessor as it now only connects a
consuming and a producing GlTextureProcessor rather than a previous,
current, and next GlTextureProcessor.
Also use default no-op implementations of the listeners in
SingleFrameGlTextureProcessor and MediaPipeProcessor to avoid
null-checks.
PiperOrigin-RevId: 466301642
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.
PiperOrigin-RevId: 464060047
Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
The old URL doesn't correctly signal the HDR10 color info in the container.
The new URL signals ST2084 (PQ) transfer function and BT.2020 color space as expected.
PiperOrigin-RevId: 461560107
This allows the service to be switched to run in another process and the app still works the same as if it is running in the same process.
Issue: androidx/media#100
PiperOrigin-RevId: 458460005
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.
PiperOrigin-RevId: 458014932
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.
PiperOrigin-RevId: 456530718
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
This is useful for testing Transformer in the same way as it is used
in tests and to see only the real transformation time.
PiperOrigin-RevId: 456058466
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
These legacy callbacks are currently forwarded to onSetMediaUri which
will be removed in the future.
Also make sure to only call player.prepare/play after the items have
been set.
The calls to onAddQueueItem are also forwarded to onAddMediaItems to
actually allow a session to resolve these items to playable media, which
wasn't possible so far.
PiperOrigin-RevId: 453625204
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
The MediaItemFiller is not flexible enough for most realworld usages
because:
- it doesn't allow asynchronous resolution of MediaItems (e.g. to
look up URIs from a database)
- it doesn't allow to batch updates for multiple items or do more
advanced customizations (e.g. expanding a mediaId representing
a playlist to multiple items).
Both issues can be solved by passing in a list of items and
returning a ListenableFuture. The callback itself can also move
into MediaSession.Callback for consistency with the other
callbacks.
PiperOrigin-RevId: 451857319
Decode-only video frames (needed when the frame at / first frame after the
clipping start is not a key frame) need to be decoded but not passed to
the frame processor chain or encoder.
The clipping start offset needs to be removed from the frame timestamps
in the passthrough and video pipelines.
There are no changes needed for this in the audio pipeline, as it doesn't
use the input timestamps -- it uses its own timestamps derived from the
buffer sizes instead.
Also add demo option to try this out.
#minor-release
PiperOrigin-RevId: 451353609
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
* Rename (via deprecation)
`MediaItem.DrmConfiguration.Builder#forceSessionsForAudioAndVideoTracks`
to `setForceSessionsForAudioAndVideoTracks`. This is more consistent
with existing 'force' method names both in this class and in
`TrackSelectionParameters.Builder`.
* Add missing `@Nullable` annotation to the parameter for
`MediaItem.SubtitleConfiguration.Builder#setMimeType`. This annotation
is already present on the `MediaItem.SubtitleConfiguration#mimeType`
field that this setter corresponds to.
PiperOrigin-RevId: 450941336
Put cloud storage samples at the top to avoid having a sample at the top of the list where we don't control the server.
Also update labels not to mention progressive container type, as it's irrelevant for Transformer, which always transmuxes even if it doesn't transcode.
#ame-bug-week
PiperOrigin-RevId: 450403784
Immediately notify a subscribing client about the availability of child items of
a parent ID to make the client load the children.
PiperOrigin-RevId: 450396690
This listener replaces
FrameProcessorChain#getAndRethrowBackgroundExceptions.
The listener uses a new exception type FrameProcessingException
separate from TransformationException as the frame processing
components will be made reusable outside of transformer soon.
PiperOrigin-RevId: 447455746
This constant is used for https://docs.gl/es2/glVertexAttribPointer
which takes the number of components per generic vertex attribute
(meaning the size of the individual coordinate vectors here) not the
number of attributes (the number of vertices that the old constant
name referred to).
PiperOrigin-RevId: 447427241
This change splits AdvancedFrameProcessor into 4 files:
- MatrixTransformationFrameProcessor for the GlFrameProcessor
implementation
- MatrixTransformation and GlMatrixTransformation for the GlEffect
specification
- MatrixUtils for the static matrix helpers
PiperOrigin-RevId: 446236384
