473 Commits

Author SHA1 Message Date
hschlueter
81b194b28b Rename TransformerTest to TransformerEndToEndTest.
TransformerTest sounds like a unit test for Transformer but these
tests test behaviour that involves multiple stages of the pipeline.

PiperOrigin-RevId: 425378369
2022-02-01 14:16:32 +00:00
hschlueter
a49a7d72e1 Fix processing remaining FrameEditor input after decoder has ended.
If
  a) the end of stream buffer arrives with a frame rather than an
     empty buffer or
  b) processDataV29() renders several decoder output buffers to the
     FrameEditor's input Surface immediately before encountering the
     EOS flag
these frames were previously stuck in the FrameEditor's input Surface
and never fed to the encoder.

PiperOrigin-RevId: 424898820
2022-02-01 14:10:45 +00:00
hschlueter
afc10c79eb Add emulator video transcoding test that counts frames.
This test tests that all frames are processed when transcoding
video to a different sample MIME type (and that the transformation
completes successfully).

PiperOrigin-RevId: 424896014
2022-02-01 14:09:43 +00:00
claincly
d68b790077 Rollback of 2ed1deb52d
*** Original commit ***

Apply suggested AVC profile depending on the API version.

***

PiperOrigin-RevId: 424856077
2022-01-28 16:49:54 +00:00
kimvde
3a23383bc3 Fix NPE in Transformer.getProgress()
If muxerWrapper.release() was throwing an exception, the progress state
was not updated and getProgress could throw an exception.

#minor-release

PiperOrigin-RevId: 424696783
2022-01-28 08:59:27 +00:00
andrewlewis
cfbdc7ba5b Remove FOREGROUND_SERVICE permission
This shouldn't be required for transformer instrumentation tests, as they don't use a foreground service.

PiperOrigin-RevId: 424654463
2022-01-28 08:58:13 +00:00
kimvde
d6e5352285 Make sure that onTransformationError is called once
The second error is probably a consequence of the first one.

#minor-release

PiperOrigin-RevId: 424619944
2022-01-28 08:57:21 +00:00
kimvde
cf792cc576 Fix transformer stuck forever
When the decoder output buffer was partially read, a call to
Codec.getOutputBuffer() was returning the same buffer, but with the
position reset to 0. The reason was that, in
Codec.maybeDequeueAndSetOutputBuffer(), mediaCodec.getOutputBuffer()
was called with the same buffer index (L350 in old rev), even though
there was already a buffer available (outputBufferIndex >=0). This
change avoids calling mediaCodec.getOutputBuffer() if the previous
buffer has not been released.

#minor-release

PiperOrigin-RevId: 424612197
2022-01-28 08:56:01 +00:00
hschlueter
75c0896f7e Take encoder resolution fallback into account in video sample pipeline.
If the encoder picks a fallback resolution the video pipeline needs
to take this into account when configuring the frameEditor and when
setting up the fallback TransformationRequest that's passed to the
fallbackListener.

PiperOrigin-RevId: 424611290
2022-01-28 08:55:11 +00:00
claincly
2ed1deb52d Apply suggested AVC profile depending on the API version.
PiperOrigin-RevId: 424322341
2022-01-28 08:41:12 +00:00
kimvde
7b8f33e848 Various nits in Transformer
PiperOrigin-RevId: 423822317
2022-01-25 19:17:31 +00:00
hschlueter
41897f5f59 Delete old TODO.
This was resolved in a652c90483 where the
Video/AudioSamplePipeline call this method before calling
the FallbackListener.

PiperOrigin-RevId: 423284258
2022-01-25 19:10:08 +00:00
hschlueter
d277deb335 Transcode to a muxer-supported sample MIME type.
If the output sample MIME type is inferred from the input
but is not supported by the muxer, we fallback to transcoding
to a supported sample MIME type.
The audio and video renderers need to make sure not to select the PassthroughSamplePipeline for this case. Which sample MIME type
to choose is decided by the EncoderFactory.

PiperOrigin-RevId: 423272812
2022-01-25 19:06:26 +00:00
hschlueter
a652c90483 Communicate sample MIME type changes to FallbackListener.
We may fall back to a different sample MIME type because
a) the sample MIME type inferred from the input is not supported
by the muxer or b) no encoders are available for the the requested
sample MIME type.

PiperOrigin-RevId: 422849036
2022-01-25 19:01:02 +00:00
hschlueter
c566ed91ad Make defensive copies of the transformation matrix.
TransformationRequest is otherwise immutable, so if we modify the
transformationMatrix in place (done before this cl) this may cause
confusing behaviour for apps when they reuse a TransformationRequest.

PiperOrigin-RevId: 422822916
2022-01-25 18:58:29 +00:00
hschlueter
e5fde04a19 Merge muxer and encoder output format error codes.
After implementing fallback, it won't always be possible to
differentiate between muxer and encoder as the cause of an output
format not being supported.

PiperOrigin-RevId: 422780443
2022-01-25 18:53:46 +00:00
claincly
38cdbbffb4 Support encoding square videos.
PiperOrigin-RevId: 422585277
2022-01-25 18:50:48 +00:00
claincly
123a3c2731 Throw NPE instead of IAE when sampleMimeType is null.
PiperOrigin-RevId: 422550627
2022-01-25 18:49:30 +00:00
claincly
05d5937855 Use encoder output format for configuring the Encoder.
This CL implements fixing the input format to the encoder spec. Fixed
parameters include:
- MIME type
- Profile & level
- Resolution
- frame rate, and
- bitrate

PiperOrigin-RevId: 422513738
2022-01-25 18:46:46 +00:00
huangdarwin
ace1988a35 Transformer GL: Add image diff in test.
PiperOrigin-RevId: 422392959
2022-01-25 18:44:22 +00:00
huangdarwin
9ef376e4cb Transformer GL: Add pixel tests for transformation.
Rotation, translation, and scale tests on a normal video.

PiperOrigin-RevId: 422383176
2022-01-25 18:43:11 +00:00
hschlueter
a62a189b41 Revise TransformationRequest MIME type validation.
PiperOrigin-RevId: 422333929
2022-01-25 18:37:21 +00:00
hschlueter
a18b64d20d Add javadoc to TransformationRequest's public fields.
PiperOrigin-RevId: 422325859
2022-01-25 18:33:01 +00:00
hschlueter
4ab10484ad Add FallbackListener.
The app will be notified about fallback using a callback on
Transformer.Listener. Fallback may be applied separately for
the audio and video options, so an intermediate internal
FallbackListener is needed to accumulate and merge the track-specific
changes to the TransformationRequest.

PiperOrigin-RevId: 421839991
2022-01-25 18:21:56 +00:00
huangdarwin
65adbbb745 Transformer GL: Clarify variables and comments.
Simplifying and clarifying variables, and adding comments.

Tested by confirming demo-gl and demo-transformer both
correctly display videos

PiperOrigin-RevId: 421792079
2022-01-25 18:17:15 +00:00
hschlueter
4d5bf7c065 Use specific error code for exceptions during encoding/decoding.
After this change exceptions throw by MediaCodec during
encoding/decoding will result in TransformationExceptions with
ERROR_CODE_ENCODING_FAILED/ERROR_CODE_DECODING_FAILED.
Before this change ERROR_CODE_FAILED_RUNTIME_CHECK was used.

PiperOrigin-RevId: 421560396
2022-01-25 18:09:19 +00:00
huangdarwin
4ea08d5117 Transformer GL: Add pixel test instructions for physical devices
Expected images are taken on emulators, so a larger acceptable
difference from expected images must be accepted on physical devices.

PiperOrigin-RevId: 421543441
2022-01-25 18:08:12 +00:00
samrobinson
9a4ad05586 Uncomment line.
Accidentally commented out the Ignore annotation.

PiperOrigin-RevId: 421304369
2022-01-25 17:59:12 +00:00
samrobinson
aab4872fc2 Add a Builder for TransformationResult.
PiperOrigin-RevId: 421278099
2022-01-25 17:58:04 +00:00
hschlueter
725b861f54 Allow multiple Transformer listeners to be registered.
Multiple listeners can be added to Transformer and its builder.
All or specific listeners can also be removed.

PiperOrigin-RevId: 421047650
2022-01-25 17:52:29 +00:00
hschlueter
b0ae7c04d5 Remove MediaCodecAdapter dependency from Transformer.
Codec and its factories can use MediaCodec directly as for API >= 21,
the SynchronousMediaCodecAdapter methods used in Codec just correspond
to a single MediaCodec call each so there is no reason to have another
wrapping layer.

PiperOrigin-RevId: 421041177
2022-01-25 17:51:11 +00:00
hschlueter
657e8768be Update Muxer exception javadoc to match MuxerWrapper.
PiperOrigin-RevId: 421039869
2022-01-25 17:49:46 +00:00
hschlueter
6070b200ae Add error code and exception type for muxing failures.
Exceptions thrown by MediaMuxer are converted MuxerExceptions
and later to TransformationExceptions with ERROR_CODE_MUXING_FAILED.

PiperOrigin-RevId: 421033721
2022-01-25 17:46:50 +00:00
claincly
e854be69e4 Replace static method with a static field.
PiperOrigin-RevId: 420307694
2022-01-11 13:38:41 +00:00
hschlueter
3d8b1c9904 Refactor AudioSamplePipeline configuration.
The encoder and sonic are now set up in the constructor rather
than in a configuration method called from processData(). This
is more similar to VideoSamplePipeline and reduces null checks.

PiperOrigin-RevId: 420260526
2022-01-11 13:35:12 +00:00
hschlueter
68613e6a1a Misc small fixes in Transformer.
PiperOrigin-RevId: 420056876
2022-01-11 13:33:49 +00:00
hschlueter
e2c4fd80d3 Wrap PlaybackExceptions in TransformationExceptions.
PiperOrigin-RevId: 420032157
2022-01-11 13:30:09 +00:00
hschlueter
cdbcf9f6f2 Make TransformationException constructor private.
Only allowing TransformationExceptions to be created using the factory methods helps keeping error messages consistent. This is consistent with ExoPlaybackException.

PiperOrigin-RevId: 419841025
2022-01-11 13:25:05 +00:00
huangdarwin
79f471b59a Transformer GL: Fix rotation distortion by considering aspect ratio
Compensate for aspect ratio of input frames, so that they're applied on
rectangular frames instead of square normalized-device-coordinate frames.

This fixes distortion most visible when rotating any GL video 45°
(non-rectangular frames) or 90° (stretched frames)

Tested by rotating several landscape/portrait demo videos.
(Automated tests will follow in <unknown commit>)

PiperOrigin-RevId: 419619743
2022-01-05 12:15:35 +00:00
hschlueter
f60dc148f8 Refactor checking muxer support.
* Move checking that the output format is supported by the muxer
  from supportsFormat (which deals with the input format) to
  ensureConfigured.
* Add maps for the supported MIME types so that the muxer can
  return what MIME types it supports rather than just check a
  MIME type.

PiperOrigin-RevId: 419578165
2022-01-05 12:15:34 +00:00
hschlueter
1af1213154 Use TransformationException for GL errors.
PiperOrigin-RevId: 418820557
2022-01-05 12:15:27 +00:00
hschlueter
fe1ffdb959 Throw when inferred sample MIME type is not supported by the muxer.
This is better than silently dropping tracks as done previously. Later,
we will implement fallback to transcoding to a supported MIME type.

PiperOrigin-RevId: 418006258
2022-01-05 12:14:58 +00:00
huangdarwin
c37319c519 Transformer GL: Document lack of support for non-square pixels.
This may one day change, but at least for now, we don't intend
to support non-square pixels.

PiperOrigin-RevId: 417983516
2022-01-05 10:52:51 +00:00
ibaker
34a1f884dc Add MediaSource.Factory and deprecate MediaSourceFactory
This more closely matches the pattern we have for all implementations
except DefaultMediaSourceFactory (e.g. ProgressiveMediaSource.Factory)
and other factory interfaces like (Http)DataSource.Factory.

PiperOrigin-RevId: 417826803
2022-01-05 10:51:41 +00:00
andrewlewis
920d0c5842 Switch naming convention for shaders
Switch to using sentence-case naming convention but with one character prefixes for different types.

This is a no-op change.

PiperOrigin-RevId: 417791624
2022-01-05 10:47:35 +00:00
hschlueter
0e61f44d4d Add TransformationRequest.
PiperOrigin-RevId: 417786661
2022-01-05 10:46:36 +00:00
claincly
4706d3a59f Rename MediaCodecAdapterWrapper to Codec.
Move static factories into a separate class and make it implement an interface
that will let tests customize encoder/decoder creation.

PiperOrigin-RevId: 417610825
2022-01-05 10:43:18 +00:00
hschlueter
991a22ec75 Convert PlaybackExceptions to TransformationExceptions.
Transformer uses ExoPlayer for reading input. Apps using Transformer
do not need to know this. So, PlaybackExceptions are converted to
TransformationExceptions with the same message, cause and error code.

The corresponding IO error codes are copied from PlaybackException.

PiperOrigin-RevId: 416793741
2022-01-05 10:28:27 +00:00
hschlueter
a8dbc744db Remove ExoPlaybackException dependency from sample pipelines.
Use TransformationException for codec and audio processor
initialization problems instead.

PiperOrigin-RevId: 416765510
2022-01-05 10:26:36 +00:00
hschlueter
e094ceb079 Use C.LENGTH_UNSET for resolution parameter instead of Format.NO_VALUE.
Format.NO_VALUE should only be used for Format fields.

PiperOrigin-RevId: 416646415
2022-01-05 10:24:36 +00:00