Size requires API 21. Using Pair instead will allow effects to be
used from API 18 during previewing once they are moved out of
transformer.
PiperOrigin-RevId: 463206474
(cherry picked from commit b994f8bfa06c82fbfe96044b986cd7b8d243acdd)
This is needed for applying effects to a playlist.
The effects are applied based on the presentation time of the
frame in its corresponding media item and the offset is added
back before encoding.
Each time the offset changes, end of input stream is signalled
to the texture processors. This is needed because the texture
processors can expect monotonically increasing timestamp within
the same input stream but when the offset changes, the timstamps
jump back to 0.
PiperOrigin-RevId: 462714966
(cherry picked from commit 46ab06b816d93c2a034c182552fa4483a6d82cba)
This allows us to use BT.2020 RGB linear for intermediate shaders, which also
allows us to re-enable PeriodicVignetteProcessor, which should work properly in
linear color-spaces.
Manually tested by adding a GlEffectsWrapper, and confirming that HLG HDR editing still looks correct.
PiperOrigin-RevId: 462265821
(cherry picked from commit 1ecf1eb7bfdd469fe855edf6126b2822b52ff07f)
* Introduced `useHdr` for `GlEffect#toGlTextureProcessor`, so
`TextureProcessor` implementations can decide how to handle HDR.
* Creating FP16 color textures for HDR input.
Tested via manual testing, adding a no-op GlEffectWrapper to the transformation to
force use of intermediate textures, adding a linear ramp to the fragment shader,
and trying to ascertain that there's a real reduction in posterization when
switching from 4-bit to 8-bit unsigned bytes, and again from 8-bit unsigned bytes
to 16-bit floating point.
PiperOrigin-RevId: 461613117
(cherry picked from commit f67c1a73f443f411771676b761ceb03e7a0d48e4)
The old URL doesn't correctly signal the HDR10 color info in the container.
The new URL signals ST2084 (PQ) transfer function and BT.2020 color space as expected.
PiperOrigin-RevId: 461560107
(cherry picked from commit 405be80fe5aed5cbc315a128010e6e6ecd7c2c73)
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.
PiperOrigin-RevId: 458014932
(cherry picked from commit f41e6796ce39469f9fc3a844ec88725ab56d3356)
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.
PiperOrigin-RevId: 456530718
(cherry picked from commit e25bf811959baf1066ba18adc37e8bbdaad4791a)
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
(cherry picked from commit 3a966916545f92c5ce08a640c912054ede215152)
This is useful for testing Transformer in the same way as it is used
in tests and to see only the real transformation time.
PiperOrigin-RevId: 456058466
(cherry picked from commit a444bb8ca2d59650811ea3f1c1274140b0be86c6)
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
(cherry picked from commit 3f718b0d10e1d577620cca9ec76d0af38efc542c)
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
(cherry picked from commit dc668f2b59bb0df12cdbb77cb88072babe0218eb)
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
(cherry picked from commit 0b37d860d13df1ac638f5a75c7af32c840e02956)
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
(cherry picked from commit 54d44d38b6bb4a7a5dc69ed8cfd878aebe46f4b4)
This should fix running `./gradlew clean test` if MediaPipe hasn't been built, for example.
PiperOrigin-RevId: 452033698
(cherry picked from commit 14ed32c68f091328bb5feaa472b4bd359e11f41d)
Decode-only video frames (needed when the frame at / first frame after the
clipping start is not a key frame) need to be decoded but not passed to
the frame processor chain or encoder.
The clipping start offset needs to be removed from the frame timestamps
in the passthrough and video pipelines.
There are no changes needed for this in the audio pipeline, as it doesn't
use the input timestamps -- it uses its own timestamps derived from the
buffer sizes instead.
Also add demo option to try this out.
#minor-release
PiperOrigin-RevId: 451353609
Also update names of implementations to match design doc.
In follow-ups, SingleFrameGlTextureProcessor will become
an abstract implementation of a new GlTextureProcessor
interface.
Texture processor makes sense as it processes OpenGL textures.
The term frame processor will be used for something else in
follow-ups.
PiperOrigin-RevId: 451142085
* Rename (via deprecation)
`MediaItem.DrmConfiguration.Builder#forceSessionsForAudioAndVideoTracks`
to `setForceSessionsForAudioAndVideoTracks`. This is more consistent
with existing 'force' method names both in this class and in
`TrackSelectionParameters.Builder`.
* Add missing `@Nullable` annotation to the parameter for
`MediaItem.SubtitleConfiguration.Builder#setMimeType`. This annotation
is already present on the `MediaItem.SubtitleConfiguration#mimeType`
field that this setter corresponds to.
PiperOrigin-RevId: 450941336
Put cloud storage samples at the top to avoid having a sample at the top of the list where we don't control the server.
Also update labels not to mention progressive container type, as it's irrelevant for Transformer, which always transmuxes even if it doesn't transcode.
#ame-bug-week
PiperOrigin-RevId: 450403784
This listener replaces
FrameProcessorChain#getAndRethrowBackgroundExceptions.
The listener uses a new exception type FrameProcessingException
separate from TransformationException as the frame processing
components will be made reusable outside of transformer soon.
PiperOrigin-RevId: 447455746
This constant is used for https://docs.gl/es2/glVertexAttribPointer
which takes the number of components per generic vertex attribute
(meaning the size of the individual coordinate vectors here) not the
number of attributes (the number of vertices that the old constant
name referred to).
PiperOrigin-RevId: 447427241
This change splits AdvancedFrameProcessor into 4 files:
- MatrixTransformationFrameProcessor for the GlFrameProcessor
implementation
- MatrixTransformation and GlMatrixTransformation for the GlEffect
specification
- MatrixUtils for the static matrix helpers
PiperOrigin-RevId: 446236384
Developers are expected to (eventually) only use methods on
PlayerView (and not PlayerControlView) to interact with the UI
controller.
PiperOrigin-RevId: 445361488
This fixes some small niggles:
1. `inferContentType(String)` is documented to take a path, but in the
tests we're passing full URIs.
2. A `String` parameter is usually a path, but also a MIME type or an
extension. In the new methods, the meaning of a `String` parameter
is always clear from the name of the method.
3. `inferContentType(String)` is always passed an extension in
'production' code (which has to be manually prefixed with a dot).
4. `inferContentType(Uri, @Nullable String)` always ignores the Uri if
the String is non-null. IMO this logic is clearer to a reader if it's
just in-lined at the call-site.
These methods are used from the demo apps, so will be part of the stable
API.
PiperOrigin-RevId: 444826053
Add a very short (1 second) video, so that some manual tests / prototyping,
including tests for the start and end of a video, encoder selection, or
changes applied to frames, can finish quickly.
PiperOrigin-RevId: 441756901
What a minimal implementation should include is now explained in the
interface javadoc while the method name reflects what the method does.
PiperOrigin-RevId: 441432059
The new demo GlFrameProcessor is based on BitmapOverlayVideoProcessor
from the gl-demo. The demo-only GlFrameProcessor can be deleted once
Transformer supports this functionality.
PiperOrigin-RevId: 440059735
PeriodicDimmingFrameProcessor is an example of how a custom fragment
shader can be used to apply color changes that change over time.
PiperOrigin-RevId: 439840609
The matrix provider allows the transformation matrix to be updated
for each frame based on the timestamp.
The following example effects using this were added to the demo:
* a zoom-in transition for the start of the video,
* cropping a rotating rectangular frame portion,
* rotating the frame around the y-axis in 3D.
PiperOrigin-RevId: 439791592
