392 Commits

Author SHA1 Message Date
huangdarwin
20dac7cb0b FrameProcessor: Create EncoderCompatibilityFrameProcessor.
Split rotationDegrees changes to EncoderCompatibilityFrameProcessor, a new
FrameProcessor.

This removes automatic rotationDegrees adjustments from Presentation, which
allows Presentation to be used for changes before the end of a
FrameProcessorChain pipeline.

PiperOrigin-RevId: 443387226
2022-04-26 16:23:32 +01:00
andrewlewis
3867f8e005 Expand Samsung/1+ workaround to API 31 + 32
The issue with decoding getting stuck is still reproducible on `samsung/beyond1lteeea/beyond1:12/SP1A.210812.016/G973FXXUEGULB:user/release-keys`.

PiperOrigin-RevId: 443368546
2022-04-26 16:16:34 +01:00
hschlueter
e996d48b0e Don't use API 26 SurfaceTexture constructor in frame processor tests.
After this change the test will use EGL_NO_SURFACE or a pixel buffer surface if using no surface is not supported.

PiperOrigin-RevId: 443113794
2022-04-26 16:09:43 +01:00
claincly
34591976ce Apply resolution fix.
Some devices under report their resolution support, like 2144 for 2160 in
H265, 1072 for 1080 in H264. This CL only takes care of these two cases,

- reporting 1920x1080 is supported when the device reports 1920x1072, and
- reporting 3840x2160 is supported when the device reports 3840x2144

PiperOrigin-RevId: 443095042
2022-04-26 16:02:49 +01:00
samrobinson
eb72e0aa09 Add 4 different video resolutions for bitrate analysis.
PiperOrigin-RevId: 442844189
2022-04-26 15:55:53 +01:00
huangdarwin
6138a0b07f FrameProcessor: Improve PresentationFrameProcessor javadocs.
PiperOrigin-RevId: 442835160
2022-04-26 15:52:28 +01:00
hschlueter
7e76f773e6 Add context param to GlFrameProcessor initialize().
PiperOrigin-RevId: 442826391
2022-04-26 15:48:51 +01:00
hschlueter
ba5d570aca Block until FrameProcessorChain is released.
This is safer because it will prevent any future problems with
creating a new FrameProcessorChain before the previous one has
completed its async release.

From [eglDestroyContext documentation](https://www.khronos.org/registry/EGL/sdk/docs/man/html/eglDestroyContext.xhtml):
"If the EGL rendering context is not current to any thread,
eglDestroyContext destroys it immediately."
The context isn't current to any thread here because GlUtil calls
eglMakeCurrent with EGL_NO_CONTEXT before calling eglDestroyContext.
So everthing should be released once the FrameProcessorChain's
release task terminates.

PiperOrigin-RevId: 442807484
2022-04-26 15:45:26 +01:00
claincly
570769ac9a Add checks for device capability in tests.
PiperOrigin-RevId: 442751310
2022-04-26 15:38:00 +01:00
huangdarwin
96b501409d FrameProcessor: Add checks to ensure width and height are positive.
Negative and zero values should be disallowed.

PiperOrigin-RevId: 441757246
2022-04-26 15:27:44 +01:00
huangdarwin
e077c76684 FrameProcessor: Move test bitmaps to a separate directory.
* Group what's now many related test PNGs by moving them to their own directory.
* Move bitmap references to files where they're used, as each bitmap is only
used once each, except the original bitmap.

PiperOrigin-RevId: 441485489
2022-04-26 15:17:20 +01:00
hschlueter
2ad44a79e6 Clarify TransformationRequest setResolution javadoc.
Scaling and rotation using ScaleToFitFrameProcessor may change the
the resolution and aspect ratio, so defaulting to the same as the
input is no longer accurate.

PiperOrigin-RevId: 441463349
2022-04-26 15:13:49 +01:00
hschlueter
fe7e330fe8 Rename GlFrameProcessor updateProgramAndDraw to drawFrame.
What a minimal implementation should include is now explained in the
interface javadoc while the method name reflects what the method does.

PiperOrigin-RevId: 441432059
2022-04-26 15:10:15 +01:00
huangdarwin
304d579e05 FrameProcessor: Add aspect ratio changes to Presentation.
PiperOrigin-RevId: 441250773
2022-04-26 15:03:16 +01:00
olly
0ad508b14f Rename TracksInfo and TrackGroupInfo
1. TracksInfo is renamed to Tracks
2. TracksInfo.TrackGroupInfo is renamed to Tracks.Group

PiperOrigin-RevId: 441232373
2022-04-26 14:59:00 +01:00
hschlueter
3f47c1ead3 Make TransformerEndToEndTest constants private.
PiperOrigin-RevId: 441230941
2022-04-26 14:55:32 +01:00
samrobinson
3b21c7e4bc Remove transformer from inputValues json.
Transformer resolves to null when wrapped, this is a debug line
that was not removed.

PiperOrigin-RevId: 441200790
2022-04-26 14:51:52 +01:00
hschlueter
eea6fc0c13 Fix FrameProcessorChain method ordering.
signalEndOfInputStream makes more sense before isEnded and
getPendingInputFrameCount is related to registerInputFrame.

PiperOrigin-RevId: 441134418
2022-04-26 14:44:29 +01:00
samrobinson
e4d7e5e7f5 Allow suppression of AssertionError in TransformerAndroidTestRunner.
PiperOrigin-RevId: 440952335
2022-04-26 14:40:10 +01:00
samrobinson
7b78548e25 Fix incorrect test id.
PiperOrigin-RevId: 440906834
2022-04-26 14:36:34 +01:00
claincly
92303810f8 Removing using the test asset that has Mp3 audio track.
The stream with the mp3 track is added because

- We only encode to AAC
- We only encode when the source track is not AAC

Now that we have a way to force encoding, we no longer need the mp3 track.

The test asset is kept for later parameterized testing.

PiperOrigin-RevId: 440876080
2022-04-26 14:32:46 +01:00
hschlueter
1fbaa5c05d Add WorkerThread annotation to FrameProcessorChain helpers.
PiperOrigin-RevId: 440347550
2022-04-26 14:22:06 +01:00
huangdarwin
187b45bc3a FrameProcessor: Add setCrop to Presentation.
PiperOrigin-RevId: 440325693
2022-04-26 14:18:26 +01:00
claincly
7fc699e97f Change SSIM transcode MIME type to H264.
The MIME type was set to H265 to force transcoding. Now that we have an encoder
factory that forces encoding, switching back to H264 ensures the quality test
is conducted on more devices (those don't support H265 can be tested now).

However, H265 should be part of the quality test after we have proper mechanism
to skip test based on device capability.

PiperOrigin-RevId: 440132471
2022-04-26 14:14:51 +01:00
claincly
1f265a4fac Add trying 1/3 resolution in fallback.
1/4 is probably unnecessary, as 1/3 of 4k is 720, which should be supported on
all practical encoders.

PiperOrigin-RevId: 440055144
2022-04-07 17:52:33 +01:00
claincly
8709b1ef02 Change SSIM test threshold from .95 to .90.
We don't currently have enough understanding of the correlation between a
specific SSIM score and video quality. Dropping to .90 to make most tests pass.
Especially when there's no discernible difference from the videos with .9 and
.95 SSIM.

PiperOrigin-RevId: 440047551
2022-04-07 17:48:55 +01:00
claincly
61a20d5f68 Add test to query device capabilities.
PiperOrigin-RevId: 439861685
2022-04-07 17:41:32 +01:00
claincly
b5eba24e1f Preallocate buffer and use byte for Luminance in SSIM.
PiperOrigin-RevId: 439855702
2022-04-07 17:38:12 +01:00
claincly
1ea4fbb720 Add test skipping for API < 23 for EncoderPerformanceTest.
PiperOrigin-RevId: 439802597
2022-04-07 17:30:42 +01:00
hschlueter
09f9a44d0f Add matrix provider for AdvancedFrameProcessor and examples in demo.
The matrix provider allows the transformation matrix to be updated
for each frame based on the timestamp.

The following example effects using this were added to the demo:
* a zoom-in transition for the start of the video,
* cropping a rotating rectangular frame portion,
* rotating the frame around the y-axis in 3D.

PiperOrigin-RevId: 439791592
2022-04-07 17:27:17 +01:00
hschlueter
c1b3e992b2 Fix typo in AdvancedFrameProcessorTest.
PiperOrigin-RevId: 439599201
2022-04-07 17:23:31 +01:00
claincly
055c10d7b8 Add javadoc for SSIM helper.
PiperOrigin-RevId: 439332549
2022-04-07 17:09:21 +01:00
hschlueter
1cbdad34a2 Support android.opengl.Matrix in AdvancedFrameProcessor.
This allows apps to use AdvancedFrameProcessor to apply transformations
in 3D space. This functionality is not used in transformer otherwise.

PiperOrigin-RevId: 439313406
2022-04-07 17:06:00 +01:00
claincly
3d93484402 Add test to evaluate performance related encoding parameters.
PiperOrigin-RevId: 439268235
2022-04-07 17:01:55 +01:00
hschlueter
a5180432ae Merge GlFrameProcessor#setInputSize() and initialize().
PiperOrigin-RevId: 439266087
2022-04-07 16:54:36 +01:00
hschlueter
bba102a467 Add ExternalCopyFrameProcessor to frameProcessors list.
PiperOrigin-RevId: 438847583
2022-04-07 16:47:08 +01:00
hschlueter
61ceae67e3 Move FrameProcessorChain OpenGL setup to factory method.
The encoder surface is no longer needed for the OpenGL setup and frame
processor initialization, as a placeholder surface is used instead. So
all of the setup can now be done in the factory method.

PiperOrigin-RevId: 438844450
2022-04-07 16:43:49 +01:00
samrobinson
6455f6e92f Add the frame count to TransformationResult.
Calculate throughputFps for TransformationTestResult.

PiperOrigin-RevId: 438817440
2022-04-07 16:37:04 +01:00
samrobinson
9fca474027 Add file logging for skipping instrumentation tests.
PiperOrigin-RevId: 438808231
2022-04-07 16:28:58 +01:00
hschlueter
0370e05395 Add FrameProcessorChain factory method and make constructor private.
PiperOrigin-RevId: 438804850
2022-04-07 16:21:44 +01:00
hschlueter
63c42b79d4 Store max timestamp rather than last written timestamp per track.
This allows the MuxerWrapper to keep using trackTypeToTimeUs for
calculating the video duration but slightly changes the meaning of
its interleaving constraints.

PiperOrigin-RevId: 438780686
2022-04-07 16:18:24 +01:00
claincly
30189f70df Move a commonly used encoder factory mode to util.
The encoder factory will be used in other tests.

PiperOrigin-RevId: 438552381
2022-04-07 16:04:28 +01:00
hschlueter
36376eb941 Split configureOutputSize into setInputSize and getOutputSize.
This makes it easier (smaller CL diff) to merge output size
configuration and initialize() in a follow-up.

PiperOrigin-RevId: 438543247
2022-04-07 15:57:30 +01:00
samrobinson
ef99d28a48 Re-use test runner in each loop of RepeatedTranscodeTransformationTest
PiperOrigin-RevId: 438542239
2022-04-07 15:54:04 +01:00
hschlueter
491de6991a Use placeholder surface to configure OpenGL and frame processors.
The placeholder surface is either EGL_NO_SURFACE or a 1x1 pbuffer
depending on whether the device supports EGL_KHR_surfaceless_context.

PiperOrigin-RevId: 438541846
2022-04-07 15:50:44 +01:00
hschlueter
775681c4eb glClear in FrameProcessorChain so the GlFrameProcessors don't have to.
Since the output textures and surfaces are managed by the
FrameProcessorChain, clearing them there makes sense.
This is also less error-prone as it might not be obvious to
someone implementing a GlFrameProcessor that they need to
glClear. (Clearing twice won't cause any problems.)

PiperOrigin-RevId: 438532247
2022-04-07 15:47:15 +01:00
samrobinson
5a43263782 Move Json helper methods to AndroidTestUtil.
PiperOrigin-RevId: 438253138
2022-04-07 15:33:39 +01:00
samrobinson
082c039b50 Add support for analyzing bitrates across devices.
Allows for input values to be propagated to the analysis file.

PiperOrigin-RevId: 438030322
2022-04-07 15:30:09 +01:00
hschlueter
d4a9e3d966 Use microseconds not nanoseconds for GlFrameProcessor.
This requires an additional nanos to micros conversion because
the SurfaceTexture uses nanos. But as the timestamps from the
MediaCodec decoder (propagated in DefaultCodec#releaseOutputBuffer) are
in microseconds no precision is lost here.

Also add test that checks output video duration.

PiperOrigin-RevId: 438010490
2022-04-07 15:23:24 +01:00
hschlueter
d609a7ffda Reword MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE javadoc.
MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE was copied from a test
class, but BitmapTestUtil isn't a test. So the javadoc needs
rewording to reflect that.

PiperOrigin-RevId: 438001833
2022-04-07 15:16:32 +01:00