Although MediaCodec claims supporting float frame rate, encoder init failed on
API21 Nexus 5. Since it's just a performance hint to the codec, it's OK to
generalize it to other API versions.
PiperOrigin-RevId: 458434650
(cherry picked from commit 1f47fa832c6228a5545d8a15c9c09fe76c17dfc5)
- Improve variable naming to include time units for clarity
- Fix existing timestamp calculations to respect time units as well as track tempo (default values for now)
- Ensure the synthesizer produces PCM for the correct amount of time (including gaps between commands).
PiperOrigin-RevId: 458428243
(cherry picked from commit da0a208b89a7914dae577900c990b0f36c23df14)
The GlEffectsFrameProcessor that will be part of the effects module
uses the DebugViewProvider. So it does not make sense for it
to be an inner interface of Transformer.
PiperOrigin-RevId: 458014932
(cherry picked from commit cd0e5b99de9577ff1af0da15a9ea6c9041f38361)
The FinalMatrixTransformationProcessorWrapper ensures that the
surface is only replaced when it is not being rendered to and vice
versa.
PiperOrigin-RevId: 458007639
(cherry picked from commit 234015cb952fbdaf274f1fa9005ddc0d2b4ba17a)
The outputHeight in the TransformationRequest is the height of
the frame as it would be displayed (i.e., after applying any
rotation specified in the format). So pass-through should only
be used if the requested outputHeight matches the input
format's height after applying the rotation.
PiperOrigin-RevId: 457934867
(cherry picked from commit 87beb273e464d291423ccb4129c83f36195343f1)
pixelWidthHeightRatio is now passed to setInputFrameInfo instead of
the factory.
PiperOrigin-RevId: 457696703
(cherry picked from commit b7b5f20e592bee772772eb1b009c7f5a96077520)
`MetadataRenderer` is updated to output `Metadata` with its presentation time, in microseconds.
PiperOrigin-RevId: 457444718
(cherry picked from commit b2831d8559713b757feb3f15a2c828db0e5ecd7c)
This will be useful for downgrading to a lower resolution during
a slow preview and for processing slide-shows once sequential
multi-asset editing is supported.
PiperOrigin-RevId: 457017255
(cherry picked from commit 30c52c58f0772e43ca4e4639120d95f41ea8a9b5)
videoEncoderFormatUnsupported_completesWithError() has recently
been flaky on API 31 emulators on presubmit because a different
exception than the expected exception is thrown.
This disables it on those emulators to reduce testing noise
until the underlying problem is investigated and resolved.
PiperOrigin-RevId: 456765512
(cherry picked from commit 938d3c2e5b6f016f7075a3d2d162d37c1951d825)
This change is just renaming. There is no functional change intended.
The FrameProcessor interface will be created in a follow-up.
PiperOrigin-RevId: 456741628
(cherry picked from commit 709224fb1ed3a1dab1433a5dc7fbb33a04eecaae)
`TextRenderer` is updated to output `CueGroup`, which contains the presentation time of the cues, in microseconds.
PiperOrigin-RevId: 456531399
(cherry picked from commit 74d61bbffb834cdb50d003a8de66af0e094ef702)
After this change GlEffects can use any GlTextureProcessor not just
SingleFrameGlTextureProcessor.
MediaPipeProcessor now implements GlTextureProcessor directly which
allows it to reuse MediaPipe's output texture for its output texture
and avoids an extra copy shader step.
PiperOrigin-RevId: 456530718
(cherry picked from commit 69ab79418ef21033ca25cebdf5a5e80752818ab5)
After this change, FrameProcessorChain chains any GlTextureProcessors
instead of only SingleFrameGlTextureProcessors.
The GlTextureProcessors are chained in a bidirectional manner using
ChainingGlTextureProcessorListener to feed input and output related
events forward and release events backwards.
PiperOrigin-RevId: 456478414
(cherry picked from commit 555ab97e34d6586d38a9979bc0595a4818b7ecb8)
This is useful for testing Transformer in the same way as it is used
in tests and to see only the real transformation time.
PiperOrigin-RevId: 456058466
(cherry picked from commit 931562c24380ff8bd1f8fad6b2c0bb3581dc65e5)
In follow-ups the FrameProcessorChain will set an instance of this
listener for each GlTextureProcessor to chain it with its previous
and next GlTextureProcesssor.
PiperOrigin-RevId: 455628942
(cherry picked from commit 981baae7095ef34c548df600bfbea1f7765dbb9e)
The wrapper
* catches exceptions for each task and notifies the
listener (this will be used more in follow-ups when processFrame
is split into lots of listeners and callbacks),
* removes finished tasks from the queue and signals any exceptions
that occurred to the listener each time a new task is executed.
PiperOrigin-RevId: 455345184
(cherry picked from commit ee847d92c50c8d2929da287e1103110149d78170)
This change adds a SurfaceProvider interface which is necessary to
allow for texture processors whose output size becomes available
asynchronously in follow-ups.
VTSP's implementation of this interface wraps the encoder and provides
its input surface together with the output frame width, height, and
orientation as used for encoder configuration.
The FrameProcessorChain converts the output frames to the provided
orientation and resolution using a ScaleToFitTransformation and
Presentation replacing EncoderCompatibilityTransformation.
PiperOrigin-RevId: 455112598
(cherry picked from commit ea7f1ca1e32f19131cec690c6934d82c81a2d6d0)
- Fixed MidiExtractor state issues which caused seeking to behave unexpectedly. Ensures the extractor is now always in the file loading state after returning RESULT_END_OF_INPUT.
- Fixed an infinite loop in MidiExtractor caused by the file data array having an initial size of 0. The extractor attempted to increase the capacity of the array by using this size of 0 in it's calculations.
PiperOrigin-RevId: 455107511
(cherry picked from commit 84c43f855fd60f96b5f415dc316c1333cfdec704)
to support enabling this as a compile-time error.
More information: go/lsc-extends-object
PiperOrigin-RevId: 454582570
(cherry picked from commit ab56f865ea076ac3cc9eb1ad5aff6000136caf80)
The native GL errors are in an arbitrary order according to
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glGetError.xml
This means any of them could contain the most useful message
so it's better to use all for the GlException's message rather than
picking the last message and logging all others.
PiperOrigin-RevId: 454130460
(cherry picked from commit d6a6b44715c15be8348b4a30664254e57d79036a)
Based on
https://developer.android.com/reference/android/media/MediaCodec#using-an-output-surface,
frame dropping behaviour depends on the target SDK version.
After this change transformer will only use
MediaFormat#KEY_ALLOW_FRAME_DROP if both the target and system SDK
version are at least 29 and default to its pre 29 behaviour where each
decoder output frame must be processed before a new one is rendered
to prevent frame dropping otherwise.
Also remove deprecated Transformer.Builder constructor without a
context and the context setter.
PiperOrigin-RevId: 453971097
(cherry picked from commit a105d033a7d44b8c3afb0e8134ad65f624caf256)
Transformer always enabled glAssertionsEnabled, so there should
be no functional change.
ExoPlayer previously disabled glAssertionsEnabled, so GlUtil logged
GlExceptions instead of throwing them. The GlExceptions are now
caught and logged by the callers so that there should also be no
functional change overall.
This change also replaces EGLSurfaceTexture#GlException with
GlUtil#GlException.
PiperOrigin-RevId: 453963741
(cherry picked from commit cc1f32d094cc4371d92b3ddab144d8965e1bc1c3)
This removes the prior restriction of needing to remember not to crop and set aspect ratio in the same Presentation.Builder, and makes each class a bit more targeted.
This is partially made feasible by the past work to merge consecutive
MatrixTransformations into a single MatrixTransformationFrameProcessor, which
ensures that there's no loss in quality between successive MatrixTransformations.
PiperOrigin-RevId: 453660582
(cherry picked from commit c5b881e0899e713a7c632da48f1cd64b1394a221)
SingleFrameGlTextureProcessor is now an abstract class containing a
default implementation of the more flexible GlTextureProcessor interface
while still exposing the same simple abstract methods for single frame
processing it previously did.
FrameProcessorChain and GlEffect will be changed to use
GlTextureProcessor in follow-ups.
PiperOrigin-RevId: 453633000
(cherry picked from commit 457f446114823831b940bf46fb256daa0ea00815)
This change makes MediaSessionPermissionTest.removeMediaItems()
non-flaky. This is a quick fix though it'd be good to refactor this
test file.
PiperOrigin-RevId: 453162764
(cherry picked from commit 34ca3bc3c3b52b8dcfc531de76bc0b34f7a89c8c)
Implementations of this interface will be able to drop or add frames,
change timestamps, accept multiple input frames before producing
output, and process frames on their own background thread.
A default implementation of this interface will be added to SingleFrameGlTextureProcessor in a follow-up.
PiperOrigin-RevId: 453159835
(cherry picked from commit 63436390de55741620ab3faccbbc504af24c2686)
This internal listener avoids wrapping the TransformationExceptions
in PlaybackExceptions that are handled via the Player.Listener and
is also used for FrameProcessingExceptions which already avoided
the PlaybackException layer previously.
This listener will also be useful in follow-ups for encoder-related
TransformationExceptions that are thrown in the SurfaceProvider that
will be called on the GL thread.
PiperOrigin-RevId: 452074575
(cherry picked from commit 35b5147eb1404698132d9b063c905202c22e31d8)
Once the more advanced GlTextureProcessor interface exists,
it will be possible to change the output size of a GlTextureProcessor
between frames. To keep the re-configuration based on the frame sizes
minimal, things indepedent of the frame size, such as the GlProgram,
can be initialized in the constructor.
PiperOrigin-RevId: 451997584
(cherry picked from commit 87ab96d352c0ffbf83d514d93193638895b61e5e)
Also rearranged release notes to correctly show when the changes were released.
PiperOrigin-RevId: 462361982
(cherry picked from commit 26c8478de667180cb8137a20ae88df285c0bb046)
The call doesn't currently reset the already loaded suppliers and
factories. Also fix the supplier loading code to use a local copy
of the current dataSourceFactory to avoid leaking an updated
instance to a later invocation.
Issue: androidx/media#116
PiperOrigin-RevId: 460721541
(cherry picked from commit adc50515e93e6fdcf303d168e8388050503c46ef)
Note: This was already reviewed in <unknown commit>. This doesn't mean we cannot apply further changes though.
PiperOrigin-RevId: 460542835
(cherry picked from commit f9a39201aafdb9b660d8c57fb8ffa051a9dc31c7)
