327 Commits

Author SHA1 Message Date
hschlueter
650f71d022 Clarify instructions for updating expected output bitmaps.
Also replace the bitmaps with bitmaps obtained by following
the instructions.

PiperOrigin-RevId: 430232369
2022-02-23 16:17:57 +00:00
andrewlewis
5867a3a29c Fix typos
PiperOrigin-RevId: 430220416
2022-02-22 17:05:01 +00:00
ibaker
805959df8b Suppress @VisibleForTesting violation in Transformer
#minor-release

PiperOrigin-RevId: 430200331
2022-02-22 17:03:56 +00:00
andrewlewis
0490839fce Suppress framework muxer lint warning
We need to access internal state to work around resources not being released on
old API versions. Add a reference to the bug about this and suppress the lint
warning.

#minor-release

PiperOrigin-RevId: 430190794
2022-02-22 17:03:06 +00:00
hschlueter
e8977d00c5 Split out ExternalCopyFrameProcessor.
All (later customizable) GlFrameProcessors after the
ExternalCopyFrameProcessor receive their input from a normal OpenGL
texture not an external texture, so they won't need to worry about
the textureTransformMatrix.

PiperOrigin-RevId: 430165652
2022-02-22 17:01:04 +00:00
samrobinson
3851ef3d69 Reduce the length of the FrameEditorDataProcessingTest TAG.
As per f2ad8ccd3c, log tags can not be longer than 23 chars.

PiperOrigin-RevId: 430032243
2022-02-22 10:34:16 +00:00
huangdarwin
3e9d45f0ea Transformer: Work around Samsung/OnePlus decoder dequeueBuffer failed.
#mse-bug-week

PiperOrigin-RevId: 430028245
2022-02-22 10:32:46 +00:00
claincly
a196b5ec43 Re-enable RepeatedTranscodeTransformationTest.
There are two major blockers to this test:

- H265 muxing is not available for API<24, so setting video mimeType to H265
  will fail on those devices.
- AMR audio encoding is buggy on some device and it's not a widely used format.

The solution: use a video that is encoded with AVC/MP3, to ensure transcoding
to AVC/AAC.

PiperOrigin-RevId: 429648598
2022-02-22 10:31:07 +00:00
samrobinson
d7adb94669 Add a new sample file to assets, which has increasing timestamps.
Re-enable tests that have no muxer support for timestamps going backwards.

Tests running on the B-frame sample will be added in a future commit.

#mse-bug-week

PiperOrigin-RevId: 429599177
2022-02-22 10:30:18 +00:00
samrobinson
4b72335d51 Add a TestTransformationResult class for additional test values.
This class will contain additional details such as frame count, once implemented.

#mse-bug-week

PiperOrigin-RevId: 429567678
2022-02-18 16:42:07 +00:00
claincly
99074f703a Improve exception message.
We use the `createForCodec` method that does not take a `MediaFormat` during
transformation, the error message always includes "no configured MediaFormat",
which is false.

PiperOrigin-RevId: 429553573
2022-02-18 16:40:19 +00:00
Ian Baker
38717ce969 Reformat some javadoc 2022-02-18 14:54:02 +00:00
samrobinson
7a5b3b3e5c Save the bitmap produced by the FrameEditorDataProcessingTest to cache.
The bitmap can then be retrieved through ADB.

#mse-bug-week

PiperOrigin-RevId: 429293231
2022-02-17 15:02:09 +00:00
hschlueter
61354497e3 Add enable fallback option to transformer demo.
Also invert disableFallback parameter in DefaultEncoderFactory
to enableFallback. Fallback is still enabled by default.

PiperOrigin-RevId: 429253266
2022-02-17 14:59:32 +00:00
andrewlewis
b96787d374 Avoid attempting to mux out of order pre-API 25
Calling `MediaMuxer.writeSampleData` can block indefinitely on old API versions. It is better not to call this method to fail quickly with an exception rather than getting stuck.

Based on on-device testing media muxer doesn't generally handle out of order samples before API 25. There are a small number of devices where this does succeed but it seems preferable to turn this off everywhere to keep the code simple and have consistent behavior. Once we switch to in-app muxing this limitation will no longer apply.

#mse-bug-week

PiperOrigin-RevId: 429070255
2022-02-17 11:52:24 +00:00
samrobinson
1af841698f Add exception information to AndroidTest analysis file.
Tested:
  Verified that the additional information is available through
  instrumentation tests, as well as via manual testing.

#mse-bug-week

PiperOrigin-RevId: 429038695
2022-02-17 11:52:23 +00:00
andrewlewis
677c5dc6d4 Skip tests requiring out of order muxing pre API 25
PiperOrigin-RevId: 429029496
2022-02-17 11:52:23 +00:00
kimvde
f8d8bfb664 Handle player release timing out in transformer
- The resources were released twice before, which is not necessary since
  the MSG_RELEASE message is already in the internal player queue.
- The demo app was failing because the stop watch was stopped in
  onTransformationError after being reset.

#minor-release
#mse-bug-week

PiperOrigin-RevId: 428794426
2022-02-17 11:52:23 +00:00
hschlueter
3cd23de1cb Split GlFrameProcessor out of FrameEditor.
The GlFrameProcessor handles everything related to the GLSL program,
the FrameEditor manages the GL context and the data flow including
the input SurfaceTexture and output EGLSurface.

This will be split up further in follow-up CLs so that
GlFrameProcessors can be chained. At this CL, there are no
functional changes intended.

PiperOrigin-RevId: 428779179
2022-02-17 11:52:23 +00:00
hschlueter
9370665b79 Track presentation timestamps in FrameEditor.
PiperOrigin-RevId: 428763554
2022-02-17 11:52:23 +00:00
huangdarwin
78b461f94c Transformer GL: Update TODOs.
Document some suboptimal things, and remove TODOs that were
fixed already or no longer relevant. Comment-only change.

PiperOrigin-RevId: 428749344
2022-02-17 11:52:23 +00:00
hschlueter
4dae8c75a8 Remove old TODO.
PiperOrigin-RevId: 428732950
2022-02-17 11:52:23 +00:00
hschlueter
015592f534 Add glClear call before drawing.
PiperOrigin-RevId: 428492031
2022-02-17 10:36:22 +00:00
hschlueter
40a5c01275 Add helpers for FBOs in GlUtil.
FBOs will be used in follow-up CLs to chain multiple GL
shader programs in Transformer.

PiperOrigin-RevId: 428016050
2022-02-17 10:29:06 +00:00
huangdarwin
b5ed01d479 Transformer GL: Implement auto-scaling to preserve input frame.
PiperOrigin-RevId: 427982223
2022-02-17 10:21:59 +00:00
kimvde
8b180eb040 Remove Transformer deprecated methods that were never released
#minor-release

PiperOrigin-RevId: 427965501
2022-02-17 10:21:09 +00:00
claincly
ba3558c0c6 Disable setting profile/level for API level < 24.
We have seen devices running on API21/23 fail transcoding because of setting
encoding profile/level.

Some devices (ale-123/nexus 7) on API21 returns ENOSYS (Function not
implemented) when being configured with a profile setting. (although API21
introduced the capability of setting encoding profile)

Some devices (nexus 5) on API23 fails configuration with a specific parameter
set, despite advertising support for it.

Not setting the baseline profile has no effect on encoding, because when not
set, the encoding will pick a suitable profile to use. Since baseline is
the lowest possible profile, the auto-picked value can't be worse than
baseline.

Ref: b/218696352
PiperOrigin-RevId: 427792124
2022-02-17 10:16:23 +00:00
claincly
b9cb87153f Add logging for exceptions thrown while configuring MediaCodec.
PiperOrigin-RevId: 427728320
2022-02-17 10:15:32 +00:00
claincly
12ce9f7c0f Fix the logic to get the max supported encoding level.
On some Android devices, the return value of

```
MediaCodecInfo.getCapabilitiesForType(mimeType).profileLevels
```

contains one entry for each encoding profile, like <profile, maxSupportedLevel>

but on some other devices, there are multiple entries for the same profile,

like <HIGH_PROFILE, LEVEL1>, <HIGH_PROFILE, LEVEL2>, <HIGH_PROFILE, LEVEL3>,
where we need to iterate through all the entries and find the max.

PiperOrigin-RevId: 427727030
2022-02-17 10:14:29 +00:00
kimvde
f36c94e104 Transformer: format test analysis file to JSON
PiperOrigin-RevId: 427469350
2022-02-17 10:07:50 +00:00
claincly
a41bdbad31 Make Codec an interface and introduce DefaultCodec.
PiperOrigin-RevId: 427191610
2022-02-08 17:19:42 +00:00
tonihei
287182952d Fix the position of IntDefs to match TYPE_USE
#minor-release

PiperOrigin-RevId: 427131569
2022-02-08 11:07:50 +00:00
claincly
31aa9d458d Fix profile-level setting.
On some old devices, the encoding level needs to be set with the encoding
profile, but not on newer devices.

The profile/level override is applied by following
https://developer.android.com/guide/topics/media/sharing-video

PiperOrigin-RevId: 427008536
2022-02-08 11:07:50 +00:00
olly
1f7174e731 Revert of 87420e5f9bd1671cd3068185f7358f5faba53e71
PiperOrigin-RevId: 426996878
2022-02-08 11:07:43 +00:00
samrobinson
403d92a4d4 Output from the Transformer the average audio & video bitrates.
PiperOrigin-RevId: 426956151
2022-02-08 10:27:15 +00:00
ibaker
87420e5f9b Fix the position of IntDefs to match TYPE_USE
#minor-release

PiperOrigin-RevId: 426855255
2022-02-07 10:49:31 +00:00
ibaker
01c814e2f0 Mark all public 'rarely used' IntDefs as only TYPE_USE
This is a breaking change if the annotation itself is in use in Kotlin
code. It's judged that the IntDefs in this commit are unlikely to be
referred to often in Kotlin code. This is because they're either:
- Related to esoteric parts of the library, or
- In a common part of the library but only returned from methods (and
  never passed to callback methods).

A follow-up change will fix the positions of existing usages to match
this new config.

#minor-release

PiperOrigin-RevId: 426410237
2022-02-07 10:47:50 +00:00
hschlueter
b3476e442a Disable fallback in tests that don't test fallback.
Fallback is only disabled for robolectric and instrumentation tests.
For MH tests, fallback is not disabled, as it may be needed due to
the broad range of devices available.

PiperOrigin-RevId: 426403167
2022-02-07 10:42:43 +00:00
claincly
f6baffc490 Roll forward of 2ed1deb52d.
Reason for not rolling back the rollback d68b790077: file name changed and
file content moved, the automated tool is unable to correctly apply the change.

Apply suggested AVC profile depending on the API version.

Use `AVCProfileHigh` only when there's encoder support.

PiperOrigin-RevId: 426363780
2022-02-04 14:50:44 +00:00
claincly
e2d4bd15cd Add option to disable encoder fallback.
We use the top priority encoder (sorted by EncoderSelector) and the requested
output format for encoding.

PiperOrigin-RevId: 426191800
2022-02-04 09:41:36 +00:00
hschlueter
7507da51a0 Make GlProgram an outer class.
This change makes GlUtil.Program an outer class named GlProgram,
and also moves private static helpers as well as the inner classes
Attribute and Uniform which were only used by GlUtil.Program to
GlProgram. Other static utility methods remain in GlUtil.

No functional changes intended.

PiperOrigin-RevId: 426119299
2022-02-04 09:40:51 +00:00
kimvde
ddfd79bb98 Use SpeedChangingAudioProcessor in Transformer
PiperOrigin-RevId: 426113559
2022-02-04 09:39:51 +00:00
ibaker
92a6cc10a4 Mark some unreleased IntDefs as TYPE_USE only
The longer list of targets is only necessary for backwards
compatibility with existing Kotlin code that will stop compiling
if the position of the annotation becomes 'wrong' by marking it only
TYPE_USE. Since none of these IntDefs have been released (except in
media3 alpha1) we don't need to maintain this compatibility.

Also add a comment to all the places that *do* need the longer list of
targets, in order to explain why it's there and discourage copy-pasting
when defining new IntDefs in future.

Also fix some single-element arrays to remove the array notation.

#minor-release

PiperOrigin-RevId: 426108537
2022-02-04 09:38:55 +00:00
claincly
7e873b5121 Improve MIME type fallback logic.
- The MIME type should ideally default to HEVC if there is an encoder for it.
- Next, check if AVC is supported.
- If there is no encoder for AVC, then we should pick an encoder in the list of
existing encoders instead of abandoning the transformation.

PiperOrigin-RevId: 425900638
2022-02-04 09:35:20 +00:00
andrewlewis
0de79209c4 Add comments describing shader programs
PiperOrigin-RevId: 425885803
2022-02-04 09:34:32 +00:00
samrobinson
134c33ba47 Remove the need for a test runner TransformationResult.
PiperOrigin-RevId: 425882755
2022-02-04 09:33:30 +00:00
claincly
e121502104 Use constructor in default constructor.
PiperOrigin-RevId: 425871885
2022-02-04 09:31:38 +00:00
samrobinson
f39b14f046 Improve Muxer documentation.
PiperOrigin-RevId: 425863658
2022-02-04 09:30:36 +00:00
kimvde
57277a23da Add SpeedChangingAudioProcessorTest
PiperOrigin-RevId: 425845144
2022-02-04 09:27:42 +00:00
claincly
31b247ea1a Split the implementation of Encoder/Decoder Factory
PiperOrigin-RevId: 425838647
2022-02-04 09:25:45 +00:00