715 Commits

Author SHA1 Message Date
huangdarwin
6a91b2df51 HDR: Remove comment explaining force sdr behavior with assetloaders.
Before, if the upstream AssetLoader provides HDR to the VideoSamplePipeline when
HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR is requested, the
VideoSamplePipeline would attempt to tell the AssetLoader to output SDR, which
could be accomplished via MediaCodec tone-mapping in the AssetLoader.

However, this makes an assumption of the AssetLoader implementation, and
AssetLoaders may not all implement support for decoder tone-mapping. Remove javadoc
attempting to explain how AssetLoaders (ex. custom ones) could behave.

PiperOrigin-RevId: 510956820
2023-02-24 09:39:08 +00:00
samrobinson
363b85226e Pass AudioFormat to ASP#computeNextEncoderInputBufferTimeUs.
This method uses sampleRate, channelCount and pcmEncoding, so passing
AudioFormat is easier.

This will lead into a future change that builds the
encoderInputAudioFormat from encoder.getConfigurationFormat()

PiperOrigin-RevId: 510956177
2023-02-24 09:35:35 +00:00
huangdarwin
540f5d681d Effect: Rename to ScaleAndRotateTransformation
Rename ScaleToFitTransformation to ScaleAndRotateTransformation.

This better represents the operations that can be accomplished using this
effect. The name was originally named ScaleToFit* because it's not obvious how
to scale to fit using OpenGL, and this effect handled the scaling to fit in a way that no other MatrixTransformations did.

However, it's hard to discover how to rotate when skimming names of effects, so
it's probably more useful to convey that this effect rotates, than that it
scales to fit.

PiperOrigin-RevId: 510480078
2023-02-24 09:28:37 +00:00
huangdarwin
84fe683ca3 HDR: Add colorinfo to exception logs.
Format.toString unfortunately doesn't log colorInfo, and as Format holds a very
large set of values, it's unclear that it should. ColorInfo is useful for codec
exceptions though, so log this in ExportException.createForCodec.

PiperOrigin-RevId: 510475520
2023-02-24 09:25:07 +00:00
andrewlewis
bd4a5ecf6a Align test codec caps check with real implementation
PiperOrigin-RevId: 510437262
2023-02-24 09:11:12 +00:00
kimvde
2163cfb24a Move generateSilentAudio to Composition
Also rename to forceAudioTrack

PiperOrigin-RevId: 510394620
2023-02-17 14:22:50 +00:00
huangdarwin
0cb5c58834 HDR: Catch test util decoder support error.
Otherwise, a lack of HDR decoding support will result in the tests checking output files for HDR output, like HdrEditingTest.transform_noRequestedTranscode_hdr10File_transformsOrThrows, failing.

PiperOrigin-RevId: 510213020
2023-02-17 13:57:07 +00:00
samrobinson
9ca1122bed Remove space in exception message.
This space was added by mistake in a prior CL of mine.

PiperOrigin-RevId: 510157363
2023-02-17 13:52:22 +00:00
huangdarwin
aa7abd5518 HDR: Add comment explaining force-SDR and AssetLoader interaction.
PiperOrigin-RevId: 510155318
2023-02-17 13:48:00 +00:00
huangdarwin
7d54f24473 HDR: Put force HDR as SDR into AssetLoader
This allows us to fix usage of HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR.

Before, this was checked in the VideoSamplePipeline, which no longer decides on the decoder configuration input format.

PiperOrigin-RevId: 510142097
2023-02-17 13:43:42 +00:00
kimvde
4cc5080db0 Rename occurrences of transform in lib-transformer
PiperOrigin-RevId: 510118760
2023-02-17 13:39:21 +00:00
andrewlewis
37768ed099 Remove Pixel watch check
It's unlikely anyone will try to use Transformer on watches.

PiperOrigin-RevId: 510115645
2023-02-17 13:34:56 +00:00
andrewlewis
ab2cabe3da Check capabilities for more tests
Some of the test methods in `TransformationTest` don't check
capabilities. Add a check just scoped to decoding (not checking
encoding, but the default encoder factory may fall back).

PiperOrigin-RevId: 510088562
2023-02-17 13:25:57 +00:00
kimvde
3f84c86977 Rename TransformationTest to ExportTest
PiperOrigin-RevId: 510087543
2023-02-17 13:21:35 +00:00
kimvde
774b44c6e2 Rename TransformationException to ExportException
PiperOrigin-RevId: 510062674
2023-02-17 13:12:44 +00:00
huangdarwin
413f61b96d Effect: Rename FrameProcessor
Rename FrameProcessor to VideoFrameProcessor, and GlEffectsFrameProcessor to
DefaultVideoFrameProcessor.

Most changes are semi-mechanical, semi-manual find-replace, preserving case:
* "FrameProc" -> "VideoFrameProc" (ex. FrameProcessor -> VideoFrameProcessor, and
   FrameProcessingException -> VideoFrameProcessingException)
* "GlEffectsVideoFrameProc" -> "DefaultVideoFrameProc"

PiperOrigin-RevId: 509887384
2023-02-17 13:04:02 +00:00
andrewlewis
3e5ae92bc6 Fix some minor nits
PiperOrigin-RevId: 509879029
2023-02-17 12:59:36 +00:00
tofunmi
5168a35654 Change FrameProcessor.create() inputTrackType parameter to a boolean
PiperOrigin-RevId: 509808913
2023-02-15 18:28:54 +00:00
andrewlewis
6d9baa6e99 Skip tests if muxing is unsupported
PiperOrigin-RevId: 509802784
2023-02-15 18:24:47 +00:00
samrobinson
7497b1e003 Use MediaFormatUtil for creating MediaFormat from Format.
PiperOrigin-RevId: 509785247
2023-02-15 12:27:43 +00:00
samrobinson
bcbe3f1f80 Clarify the mime type used in EncoderUtilTest.
PiperOrigin-RevId: 509750806
2023-02-15 10:45:30 +00:00
samrobinson
d91afa063a Align ASP and VSP on findSupportedMimeTypeForEncoder/Muxer logic.
* Moved the logic to SamplePipeline.
* Pass the requested values via Format.
* Moved exception throwing inside the methods.
* Build up the mimeTypesToCheck as a set - removing possible duplicate
checks.
* Simplified logic that calls the findSupportedMimeType method.
* Improved javadoc.

PiperOrigin-RevId: 509594062
2023-02-15 10:41:50 +00:00
samrobinson
ccd8856dac Preserve aspect ratio when scaling resolution is insufficient.
PiperOrigin-RevId: 509546771
2023-02-15 10:38:05 +00:00
claincly
efcb7683e5 Use ColorInfo.Builder in transformer and common.
Because the ColorInfo constructor is deprecated.

PiperOrigin-RevId: 509468663
2023-02-14 10:28:33 +00:00
samrobinson
dcd1f6bc6b Generalize getSupportedEncodersForHdr to return List<MediaCodecInfo>.
This now matches EncoderUtil.getSupportedEncoders return type.

PiperOrigin-RevId: 509222078
2023-02-13 15:33:02 +00:00
huangdarwin
e159741a95 Test: Move FileUtil to mh from mh.analysis.
FileUtil is only ever used in mh, and never in mh.analysis

PiperOrigin-RevId: 509206721
2023-02-13 14:12:46 +00:00
huangdarwin
15f0865d62 HDR: Remove HDR-specific error codes.
It may be confusing for 3P apps, for us to have separate error codes for (1) if
encoding HDR is not supported at all by the device, and (2) if encoding the
format, which happens to be HDR, is not supported by the device. Instead, we can
communicate this in the error message.

PiperOrigin-RevId: 509188666
2023-02-13 12:22:55 +00:00
kimvde
07ff48165a Handle when some MediaItems require transcoding but others don't
- For single-asset, the behavior stays the same. Transcode if and only
  if it's necessary,
- For constrained multi-asset, always transcode, except if the setter to
  transmux is set. This is to avoid failing if a MediaItem that doesn't
  require transcoding is followed by a MediaItem that does require
  transcoding.

PiperOrigin-RevId: 508097798
2023-02-14 16:02:45 +00:00
kimvde
45c42ed373 Add an API entry point to pass a Composition
PiperOrigin-RevId: 508031337
2023-02-08 15:50:43 +00:00
huangdarwin
01082a48f1 Effect: Create GlEffect.isNoOp to simplify Transformer.
This will also allow us to skip transcodes in the future for other effects, like
cropping, color effects, etc.

PiperOrigin-RevId: 507765618
2023-02-08 15:34:42 +00:00
kimvde
f39ae692c8 Rename TransformationResult to ExportResult
Also replace some usages of deprecated Transformer listeners with the new
ones.

PiperOrigin-RevId: 507743860
2023-02-08 15:30:40 +00:00
tofunmi
af259054af Update CompositeAssetLoader to override queueInputBitmap
PiperOrigin-RevId: 507515587
2023-02-08 15:14:39 +00:00
kimvde
4631105fb3 Rename startTransformation to start
Also update some Javadoc in Transformer

PiperOrigin-RevId: 507395956
2023-02-08 14:58:38 +00:00
tofunmi
c32794f0cc Update sample pipelines and frame processors to handle image input.
PiperOrigin-RevId: 506965394
2023-02-08 14:54:39 +00:00
huangdarwin
7684949fa4 HDR: Move codec errors from VideoSamplePipeline
It doesn't actually make sense for them to be placed in the Transformer, because the error's root causes are actually only in codecs. Also, a few codec errors were
repeated, so deduplicate these instances

PiperOrigin-RevId: 506937695
2023-02-08 14:50:44 +00:00
tofunmi
d4db00e51a Update FrameProcessor.create() to accept an input track type.
Based on [this conversation thread](https://chat.google.com/room/AAAA--f88ao/76Rem_cRCK8), I've opted to update the existing FrameProcessor.create() rather than deprecate it, as it is unlikely to be in use by apps outside google3.

PiperOrigin-RevId: 506920930
2023-02-08 14:46:49 +00:00
huangdarwin
557b7dcc71 Demo: Request READ_MEDIA_VIDEO on API 33+
More info in: https://developer.android.com/about/versions/13/behavior-changes-13#granular-media-permissions

PiperOrigin-RevId: 506908079
2023-02-08 14:42:52 +00:00
kimvde
87ff3157c9 Update Composition structure to take List instead of ImmutableList
This is more flexible for apps and more consitent with the Player API.

PiperOrigin-RevId: 506901645
2023-02-08 14:38:57 +00:00
samrobinson
73d4b67e46 Expand createNoSupportedMimeTypeException for HDR ColorInfo case.
PiperOrigin-RevId: 506900817
2023-02-08 14:35:02 +00:00
kimvde
90de454e1e Adapt TransformationResult for multi-asset
PiperOrigin-RevId: 506898392
2023-02-08 14:31:02 +00:00
samrobinson
937fcf9c4a Refactor TransformationException.createForCodec method overloads.
* Overload added `(cause, errorCode, isVideo, isDecoder, details)`,
where `details` is a string of values to be added to the error message
of the `TransformationException`.
* Overload with `MediaFormat` and `mediaCodecName` moved to
`DefaultCodec`, because all usages of that overload were from
`DefaultCodec`, and this allows a simplified API because of internally
stored values.
* `mediaCodecName` removed from overload that takes a `Format`.
* Reordered `createForCodec` parameters.

PiperOrigin-RevId: 506895268
2023-02-08 14:27:04 +00:00
kimvde
83074c0dcd Rename Transformer callbacks
PiperOrigin-RevId: 506890459
2023-02-08 14:23:02 +00:00
samrobinson
eb4b6f812c Implement a ChannelMixingAudioProcessor.
PiperOrigin-RevId: 506886903
2023-02-08 14:19:05 +00:00
claincly
c09904e143 Prefer hardware encoders in EncoderSelector.
In encoding small and odd-numbered resolutions, like `316x61` ([this image](https://upload.wikimedia.org/wikipedia/commons/6/65/100winners.png)), the current fallback logic prefers a software encoder to hardware ones. The assumption was, the encoder factory applies the encoder size alignment and changes the resolution to `316x60` for SW encoders and `316x64` for HW ones. SW encoders is selected because the supported resolution 60 is closer to requested 61, than the hardware supported 64.

This change changes the default encoder selection process to only expose hardware encoders if there is any.

PiperOrigin-RevId: 506879983
2023-02-08 14:15:03 +00:00
kimvde
14210d3df3 Rename inputFormat to firstInputFormat in multiple places
With multi-asset, the sample pipelines can process more than one
MediaItem. The renaming makes it clear that the format passed to the
SamplePipeline constructors is the one corresponding to the first
MediaItem. Indeed, the first format is the one used to configure the
SamplePipelines.

PiperOrigin-RevId: 506879260
2023-02-08 14:11:01 +00:00
samrobinson
bfee268e3f Turn off ExoPlayer metrics reporting when used by Transformer.
This avoids affecting ExoPlayer metrics with non-typical usage.

PiperOrigin-RevId: 506878231
2023-02-08 14:06:59 +00:00
tofunmi
8a8fd8dbe5 Update duration & frame rate javadoc in EditedMediaItem
PiperOrigin-RevId: 506874799
2023-02-08 14:03:02 +00:00
kimvde
3635fe7abe Add MediaItem change listener
PiperOrigin-RevId: 506868341
2023-02-08 13:59:03 +00:00
kimvde
23a544025d Use CompositeAssetLoader in Transformer
PiperOrigin-RevId: 506863538
2023-02-08 13:55:01 +00:00
tofunmi
8121709605 Add ImageAssetLoader and ImageConfiguration.
PiperOrigin-RevId: 506619637
2023-02-08 13:46:57 +00:00