278 Commits

Author SHA1 Message Date
huangdarwin
29e32540de FrameProcessor: Move test bitmaps to a separate directory.
* Group what's now many related test PNGs by moving them to their own directory.
* Move bitmap references to files where they're used, as each bitmap is only
used once each, except the original bitmap.

PiperOrigin-RevId: 441485489
2022-04-26 14:36:31 +01:00
hschlueter
22e016e851 Clarify TransformationRequest setResolution javadoc.
Scaling and rotation using ScaleToFitFrameProcessor may change the
the resolution and aspect ratio, so defaulting to the same as the
input is no longer accurate.

PiperOrigin-RevId: 441463349
2022-04-26 14:35:12 +01:00
hschlueter
f25c912c72 Rename GlFrameProcessor updateProgramAndDraw to drawFrame.
What a minimal implementation should include is now explained in the
interface javadoc while the method name reflects what the method does.

PiperOrigin-RevId: 441432059
2022-04-26 14:34:11 +01:00
huangdarwin
34f014ce5c FrameProcessor: Add aspect ratio changes to Presentation.
PiperOrigin-RevId: 441250773
2022-04-26 14:31:14 +01:00
olly
83daa052cb Rename TracksInfo and TrackGroupInfo
1. TracksInfo is renamed to Tracks
2. TracksInfo.TrackGroupInfo is renamed to Tracks.Group

PiperOrigin-RevId: 441232373
2022-04-26 14:30:14 +01:00
hschlueter
af8435816a Make TransformerEndToEndTest constants private.
PiperOrigin-RevId: 441230941
2022-04-26 14:29:19 +01:00
samrobinson
36c5b9407c Remove transformer from inputValues json.
Transformer resolves to null when wrapped, this is a debug line
that was not removed.

PiperOrigin-RevId: 441200790
2022-04-26 14:28:13 +01:00
hschlueter
9b4aba33a3 Fix FrameProcessorChain method ordering.
signalEndOfInputStream makes more sense before isEnded and
getPendingInputFrameCount is related to registerInputFrame.

PiperOrigin-RevId: 441134418
2022-04-26 14:26:17 +01:00
samrobinson
0a16813f1c Allow suppression of AssertionError in TransformerAndroidTestRunner.
PiperOrigin-RevId: 440952335
2022-04-26 14:25:15 +01:00
samrobinson
8521d48f86 Fix incorrect test id.
PiperOrigin-RevId: 440906834
2022-04-26 14:24:21 +01:00
claincly
803cf64a06 Removing using the test asset that has Mp3 audio track.
The stream with the mp3 track is added because

- We only encode to AAC
- We only encode when the source track is not AAC

Now that we have a way to force encoding, we no longer need the mp3 track.

The test asset is kept for later parameterized testing.

PiperOrigin-RevId: 440876080
2022-04-26 14:23:26 +01:00
hschlueter
20547c3392 Add WorkerThread annotation to FrameProcessorChain helpers.
PiperOrigin-RevId: 440347550
2022-04-26 14:20:31 +01:00
huangdarwin
8a7fcb4eae FrameProcessor: Add setCrop to Presentation.
PiperOrigin-RevId: 440325693
2022-04-26 14:19:38 +01:00
claincly
1ca80741e5 Change SSIM transcode MIME type to H264.
The MIME type was set to H265 to force transcoding. Now that we have an encoder
factory that forces encoding, switching back to H264 ensures the quality test
is conducted on more devices (those don't support H265 can be tested now).

However, H265 should be part of the quality test after we have proper mechanism
to skip test based on device capability.

PiperOrigin-RevId: 440132471
2022-04-26 14:18:35 +01:00
claincly
7d0034748f Add trying 1/3 resolution in fallback.
1/4 is probably unnecessary, as 1/3 of 4k is 720, which should be supported on
all practical encoders.

PiperOrigin-RevId: 440055144
2022-04-26 14:14:33 +01:00
claincly
a2d92ec623 Change SSIM test threshold from .95 to .90.
We don't currently have enough understanding of the correlation between a
specific SSIM score and video quality. Dropping to .90 to make most tests pass.
Especially when there's no discernible difference from the videos with .9 and
.95 SSIM.

PiperOrigin-RevId: 440047551
2022-04-26 14:13:31 +01:00
claincly
1da7fd8956 Add test to query device capabilities.
PiperOrigin-RevId: 439861685
2022-04-26 14:11:27 +01:00
claincly
8bdce6ade6 Preallocate buffer and use byte for Luminance in SSIM.
PiperOrigin-RevId: 439855702
2022-04-26 14:09:52 +01:00
claincly
7c64b1915c Add test skipping for API < 23 for EncoderPerformanceTest.
PiperOrigin-RevId: 439802597
2022-04-26 14:07:48 +01:00
hschlueter
c235e4f447 Add matrix provider for AdvancedFrameProcessor and examples in demo.
The matrix provider allows the transformation matrix to be updated
for each frame based on the timestamp.

The following example effects using this were added to the demo:
* a zoom-in transition for the start of the video,
* cropping a rotating rectangular frame portion,
* rotating the frame around the y-axis in 3D.

PiperOrigin-RevId: 439791592
2022-04-26 14:06:36 +01:00
hschlueter
f75710be93 Fix typo in AdvancedFrameProcessorTest.
PiperOrigin-RevId: 439599201
2022-04-06 12:00:19 +01:00
claincly
5b258ef8ab Add javadoc for SSIM helper.
PiperOrigin-RevId: 439332549
2022-04-06 11:56:02 +01:00
hschlueter
8138a9f48f Support android.opengl.Matrix in AdvancedFrameProcessor.
This allows apps to use AdvancedFrameProcessor to apply transformations
in 3D space. This functionality is not used in transformer otherwise.

PiperOrigin-RevId: 439313406
2022-04-06 11:54:59 +01:00
claincly
5b77625582 Add test to evaluate performance related encoding parameters.
PiperOrigin-RevId: 439268235
2022-04-06 11:53:53 +01:00
hschlueter
de00030f98 Merge GlFrameProcessor#setInputSize() and initialize().
PiperOrigin-RevId: 439266087
2022-04-06 11:52:05 +01:00
hschlueter
d3931d8b96 Add ExternalCopyFrameProcessor to frameProcessors list.
PiperOrigin-RevId: 438847583
2022-04-06 11:49:31 +01:00
hschlueter
2a66c7b8f5 Move FrameProcessorChain OpenGL setup to factory method.
The encoder surface is no longer needed for the OpenGL setup and frame
processor initialization, as a placeholder surface is used instead. So
all of the setup can now be done in the factory method.

PiperOrigin-RevId: 438844450
2022-04-06 11:47:45 +01:00
samrobinson
af5386cbc1 Add the frame count to TransformationResult.
Calculate throughputFps for TransformationTestResult.

PiperOrigin-RevId: 438817440
2022-04-06 11:44:16 +01:00
samrobinson
0c6882867b Add file logging for skipping instrumentation tests.
PiperOrigin-RevId: 438808231
2022-04-06 11:43:16 +01:00
hschlueter
fa47233628 Add FrameProcessorChain factory method and make constructor private.
PiperOrigin-RevId: 438804850
2022-04-06 11:41:04 +01:00
hschlueter
6d753afc4c Store max timestamp rather than last written timestamp per track.
This allows the MuxerWrapper to keep using trackTypeToTimeUs for
calculating the video duration but slightly changes the meaning of
its interleaving constraints.

PiperOrigin-RevId: 438780686
2022-04-06 11:40:06 +01:00
claincly
12fc9d1070 Move a commonly used encoder factory mode to util.
The encoder factory will be used in other tests.

PiperOrigin-RevId: 438552381
2022-04-06 11:35:43 +01:00
hschlueter
70cffd8cca Split configureOutputSize into setInputSize and getOutputSize.
This makes it easier (smaller CL diff) to merge output size
configuration and initialize() in a follow-up.

PiperOrigin-RevId: 438543247
2022-04-06 11:34:34 +01:00
samrobinson
d8e5e2de7a Re-use test runner in each loop of RepeatedTranscodeTransformationTest
PiperOrigin-RevId: 438542239
2022-04-06 11:33:38 +01:00
hschlueter
839f342e55 Use placeholder surface to configure OpenGL and frame processors.
The placeholder surface is either EGL_NO_SURFACE or a 1x1 pbuffer
depending on whether the device supports EGL_KHR_surfaceless_context.

PiperOrigin-RevId: 438541846
2022-04-06 11:32:38 +01:00
hschlueter
aadbf3d59b glClear in FrameProcessorChain so the GlFrameProcessors don't have to.
Since the output textures and surfaces are managed by the
FrameProcessorChain, clearing them there makes sense.
This is also less error-prone as it might not be obvious to
someone implementing a GlFrameProcessor that they need to
glClear. (Clearing twice won't cause any problems.)

PiperOrigin-RevId: 438532247
2022-04-06 11:30:23 +01:00
samrobinson
01c24e4de8 Move Json helper methods to AndroidTestUtil.
PiperOrigin-RevId: 438253138
2022-04-06 11:26:29 +01:00
samrobinson
e699765df5 Add support for analyzing bitrates across devices.
Allows for input values to be propagated to the analysis file.

PiperOrigin-RevId: 438030322
2022-04-06 11:24:26 +01:00
hschlueter
d97de5b9f0 Use microseconds not nanoseconds for GlFrameProcessor.
This requires an additional nanos to micros conversion because
the SurfaceTexture uses nanos. But as the timestamps from the
MediaCodec decoder (propagated in DefaultCodec#releaseOutputBuffer) are
in microseconds no precision is lost here.

Also add test that checks output video duration.

PiperOrigin-RevId: 438010490
2022-04-06 11:22:22 +01:00
hschlueter
bd6eaab6b4 Reword MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE javadoc.
MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE was copied from a test
class, but BitmapTestUtil isn't a test. So the javadoc needs
rewording to reflect that.

PiperOrigin-RevId: 438001833
2022-04-06 11:20:17 +01:00
hschlueter
808909289c Use @linkplain for link text that doesn't match symbol name.
PiperOrigin-RevId: 437992927
2022-04-06 11:18:03 +01:00
andrewlewis
d2a9419ad3 Add support for requesting color transfer to SDR
From Android T onwards `MediaCodec` supports requesting tone-mapping down to
SDR. Add an option to request this behavior and document that it isn't
supported before T. Also add an option in the demo app to try it out.

Tested manually on a prerelease build.

PiperOrigin-RevId: 437765325
2022-04-06 11:10:06 +01:00
andrewlewis
79db98e733 Remove unnecessary initialization
PiperOrigin-RevId: 437753013
2022-04-06 11:09:03 +01:00
hschlueter
827cf51dc9 Check thread name for GL methods.
The thread name is used to verify the thread in both  createOpenGlObjectsAndInitializeFrameProcessors() and processFrame().
Also remove glThread field that was only used for this verification.

PiperOrigin-RevId: 437730804
2022-04-06 11:07:45 +01:00
hschlueter
92eb09fa6f Fix output viewport size of empty FrameProcessorChain.
Since the output size can be overridden, the viewport should be
ouputWidth/Height and NOT the ExternalCopyFrameProcessor's output size
which matches the input size.

PiperOrigin-RevId: 437256635
2022-04-06 11:01:43 +01:00
hschlueter
5f573ad3ad Remove old TODOs and reassign another TODO.
* The AdvancedFrameProcessor calls use() in updateProgramAndDraw().
* The AdvancedFrameProcessor has the same input and output dimensions.

PiperOrigin-RevId: 437231350
2022-03-25 14:10:47 +00:00
hschlueter
0f5686fe07 Allow frame processors to be set on Transformer.Builder.
Also make GlFrameProcessor, ScaleToFitFrameProcessor, and
AdvancedFrameProcessor public.

PiperOrigin-RevId: 437227388
2022-03-25 14:08:41 +00:00
hschlueter
d5056072a8 Check for GL errors after glClear and glDrawArrays.
These methods can cause errors so we should check for them.
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glClear.xml
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glDrawArrays.xml

PiperOrigin-RevId: 437218519
2022-03-25 14:07:49 +00:00
andrewlewis
117456c137 Improve throughput on devices requiring workaround
On some devices, decoding gets stuck when the number of frames pending at the
`SurfaceTexture` is too high. We added a workaround that only allows one frame
to be pending at a time. That fixed the issue, however, based on on-device
testing it seems that it's safe to queue more than one frame.

Add a method that returns a safe estimate of the number of frames that can be
pending at a time, and use this to limit the number of frames that can be
released from the decoder but not processed by the frame processor chain.

PiperOrigin-RevId: 437057075
2022-03-25 14:05:08 +00:00
hschlueter
37559deacf Configure the frame sizes in FrameProcessorChain instead of caller.
Configuring the frame sizes between frame processors is now the
FrameProcessorChain's rather than the caller's responsibility.
The caller can getOutputSize() and override it for encoder fallback
in configure().

PiperOrigin-RevId: 437048436
2022-03-25 14:04:19 +00:00