8303762: Optimize vector slice operation with constant index using VPALIGNR instruction

[jatin-bhateja]

Patch optimizes Vector. slice operation with constant index using x86 ALIGNR instruction.
It also adds a new hybrid call generator to facilitate lazy intrinsification or else perform procedural inlining to prevent call overhead and boxing penalties in case the fallback implementation expects to operate over vectors. The existing vector API-based slice implementation is now the fallback code that gets inlined in case intrinsification fails.

Idea here is to add infrastructure support to enable intrinsification of fast path for selected vector APIs, else enable inlining of fall-back implementation if it's based on vector APIs. Existing call generators like PredictedCallGenerator, used to handle bi-morphic inlining, already make use of multiple call generators to handle hit/miss scenarios for a particular receiver type. The newly added hybrid call generator is lazy and called during incremental inlining optimization. It also relieves the inline expander to handle slow paths, which can easily be implemented library side (Java).

Vector API jtreg tests pass at AVX level 2, remaining validation in progress.

Performance numbers:

System : 13th Gen Intel(R) Core(TM) i3-1315U

Baseline:
Benchmark                                                (size)   Mode  Cnt      Score   Error   Units
VectorSliceBenchmark.byteVectorSliceWithConstantIndex1     1024  thrpt    2   9444.444          ops/ms
VectorSliceBenchmark.byteVectorSliceWithConstantIndex2     1024  thrpt    2  10009.319          ops/ms
VectorSliceBenchmark.byteVectorSliceWithVariableIndex      1024  thrpt    2   9081.926          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex1      1024  thrpt    2   6085.825          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex2      1024  thrpt    2   6505.378          ops/ms
VectorSliceBenchmark.intVectorSliceWithVariableIndex       1024  thrpt    2   6204.489          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex1     1024  thrpt    2   1651.334          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex2     1024  thrpt    2   1642.784          ops/ms
VectorSliceBenchmark.longVectorSliceWithVariableIndex      1024  thrpt    2   1474.808          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex1    1024  thrpt    2  10399.394          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex2    1024  thrpt    2  10502.894          ops/ms
VectorSliceBenchmark.shortVectorSliceWithVariableIndex     1024  thrpt    2   9756.573          ops/ms

With opt:
Benchmark                                                (size)   Mode  Cnt      Score   Error   Units
VectorSliceBenchmark.byteVectorSliceWithConstantIndex1     1024  thrpt    2  34122.435          ops/ms
VectorSliceBenchmark.byteVectorSliceWithConstantIndex2     1024  thrpt    2  33281.868          ops/ms
VectorSliceBenchmark.byteVectorSliceWithVariableIndex      1024  thrpt    2   9345.154          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex1      1024  thrpt    2   8283.247          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex2      1024  thrpt    2   8510.695          ops/ms
VectorSliceBenchmark.intVectorSliceWithVariableIndex       1024  thrpt    2   5626.367          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex1     1024  thrpt    2    960.958          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex2     1024  thrpt    2   4155.801          ops/ms
VectorSliceBenchmark.longVectorSliceWithVariableIndex      1024  thrpt    2   1465.953          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex1    1024  thrpt    2  32748.061          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex2    1024  thrpt    2  33674.408          ops/ms
VectorSliceBenchmark.shortVectorSliceWithVariableIndex     1024  thrpt    2   9346.148          ops/ms

Please share your feedback.

Best Regards,
Jatin

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

• JDK-8303762: Optimize vector slice operation with constant index using VPALIGNR instruction (Enhancement - P4)

Reviewers

• Xiaohong Gong (@XiaohongGong - Committer) 🔄 Re-review required (review applies to 9625b04e)
• Eric Fang (@erifan - Author) 🔄 Re-review required (review applies to ae242926)
• Sandhya Viswanathan (@sviswa7 - Reviewer) 🔄 Re-review required (review applies to c5950031)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24104/head:pull/24104
$ git checkout pull/24104

Update a local copy of the PR:
$ git checkout pull/24104
$ git pull https://git.openjdk.org/jdk.git pull/24104/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 24104

View PR using the GUI difftool:
$ git pr show -t 24104

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24104.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back jbhateja! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@jatin-bhateja This change is no longer ready for integration - check the PR body for details.

[openjdk]

@jatin-bhateja The following labels will be automatically applied to this pull request:

• core-libs
• graal
• hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[jatin-bhateja]

/label add hotspot-compiler-dev

[openjdk]

@jatin-bhateja
The hotspot-compiler label was successfully added.

[bridgekeeper]

@jatin-bhateja This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[jatin-bhateja]

/keepalive

[openjdk]

@jatin-bhateja The pull request is being re-evaluated and the inactivity timeout has been reset.

[openjdk]

@jatin-bhateja this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8303762
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

[bridgekeeper]

@jatin-bhateja This pull request has been inactive for more than 8 weeks and will be automatically closed if another 8 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[jatin-bhateja]

Performance after AVX2 backend modifications

Benchmark                                                (size)   Mode  Cnt      Score   Error   Units
VectorSliceBenchmark.byteVectorSliceWithConstantIndex1     1024  thrpt    2  51644.530          ops/ms
VectorSliceBenchmark.byteVectorSliceWithConstantIndex2     1024  thrpt    2  48171.079          ops/ms
VectorSliceBenchmark.byteVectorSliceWithVariableIndex      1024  thrpt    2   9662.306          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex1      1024  thrpt    2  14358.347          ops/ms
VectorSliceBenchmark.intVectorSliceWithConstantIndex2      1024  thrpt    2  14619.920          ops/ms
VectorSliceBenchmark.intVectorSliceWithVariableIndex       1024  thrpt    2   6675.824          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex1     1024  thrpt    2    818.911          ops/ms
VectorSliceBenchmark.longVectorSliceWithConstantIndex2     1024  thrpt    2   4778.321          ops/ms
VectorSliceBenchmark.longVectorSliceWithVariableIndex      1024  thrpt    2   1612.264          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex1    1024  thrpt    2  35961.146          ops/ms
VectorSliceBenchmark.shortVectorSliceWithConstantIndex2    1024  thrpt    2  39072.170          ops/ms
VectorSliceBenchmark.shortVectorSliceWithVariableIndex     1024  thrpt    2  11209.685          ops/ms

[mlbridge]

Webrevs

• 21: Full - Incremental (46fcc9ac)
• 20: Full - Incremental (c5950031)
• 19: Full - Incremental (2834a027)
• 18: Full - Incremental (ad7151e9)
• 17: Full - Incremental (121c40a8)
• 16: Full (bde0c216)
• 15: Full - Incremental (2b8f0b48)
• 14: Full - Incremental (9625b04e)
• 13: Full - Incremental (444a3568)
• 12: Full (1dfff558)
• 11: Full - Incremental (ae242926)
• 10: Full - Incremental (2c7eb96d)
• 09: Full (4b807102)
• 08: Full (9da1f862)
• 07: Full - Incremental (340f1849)
• 06: Full - Incremental (70c22932)
• 05: Full - Incremental (d55fa594)
• 04: Full - Incremental (f36ae6dd)
• 03: Full - Incremental (405de56f)
• 02: Full (e7c7374b)
• 01: Full - Incremental (04be59a6)
• 00: Full (b2e93434)

[XiaohongGong]

Still LGTM!

[merykitty]

I'm still not convinced with this solution. If the pattern matching method proves itself to be not reliable, then we can proceed with an intrinsics. Otherwise, we risk introduce a change that will eventually become redundant.

[jatin-bhateja]

I'm still not convinced with this solution. If the pattern matching method proves itself to be not reliable, then we can proceed with an intrinsics. Otherwise, we risk introduce a change that will eventually become redundant.

Hi @merykitty , As discussed earlier your suggestions were incorporated in latest version of patch, idea here is not to hold an optimization in anticipation of future optimization. x86 backend changes will still be usable if at later point we decide to use complex pattern matching once TypeVect has constant information. What we have currently is generic handling which can inline any fallback after failed intrinsification attempts. Looking forward to your comments on backend part and any further improvement on existing handling.

Hi @sviswa7 , @iwanowww , May I request you to share your views / comments

[iwanowww]

I briefly looked at the patch.

First of all, I suggest to separate the logic to handle intrinsification failures. It's not specific to the proposed enhancement and will improve handling of intrinsification failures for vector operations.

Speaking of proposed approach, it aligns well with current Vector API implementation practices. I agree it would be nice to automatically detect equivalent IR shapes and transform them accordingly, but if it means hard-coding the shape of sliceTemplate into the compiler, current proposal does look well-justified.

[jatin-bhateja]

I briefly looked at the patch.

First of all, I suggest to separate the logic to handle intrinsification failures. It's not specific to the proposed enhancement and will improve handling of intrinsification failures for vector operations.

Speaking of proposed approach, it aligns well with current Vector API implementation practices. I agree it would be nice to automatically detect equivalent IR shapes and transform them accordingly, but if it means hard-coding the shape of sliceTemplate into the compiler, current proposal does look well-justified.

Thanks @iwanowww , I agree that approach to inline on intrinsic failure is generic enough and can benefit other vector operations also as it may absorb boxing penalties. For slice and un-slice since the fallback is completely written in vector APIs it will give most benefits and that is the focus of this patch.

Looking forward to your other comments on current implementation.

[sviswa7]

@jatin-bhateja I agree with @iwanowww that the PR could be split into two: One handling the intrinsification failure/fallback handling and other with vector slice optimization for x86. That might help you to get reviews on this work. I volunteer to review the x86 PR. Order wise, the fallback PR would need to get in first though.

[jatin-bhateja]

@jatin-bhateja I agree with @iwanowww that the PR could be split into two: One handling the intrinsification failure/fallback handling and other with vector slice optimization for x86. That might help you to get reviews on this work. I volunteer to review the x86 PR. Order wise, the fallback PR would need to get in first though.

Hi @sviswa7 ,
Almost all the fallback code apart from few (unslice, slice etc) use scalar operation loop to compute the result, a box created on caller side on account of failed intrinsic will not be unboxed on callee side i.e. fall back implementation. In this context, inlining the fallback will save the call overhead but not prevent boxing penalty or code bloating on callee side which may have other side effects. Which is why this PR selectively enables in lining of slice fallback which is composed of vector APIs and code for that is part of this pull request.

May I request you to kindly review the x86 backend implementation part of this pull request and share your feedback.

Best Regards

[jatin-bhateja]

Hi @iwanowww , kindly let us know your comments on current implementation.

[jatin-bhateja]

Hi @sviswa7 , your comments have been addressed, kindly verify

[jatin-bhateja]

/template append

[openjdk]

@jatin-bhateja The pull request template has been appended to the pull request body

[sviswa7]

x86 changes look good to me. You will need another review from compiler folks for the changes in the call generator to handle fallback.

[iwanowww]

It would be much simpler to review inlining-related and VectorSlice-related parts separately.

[iwanowww]

What's the motivation for a separate list? Why don't you perform fallback inlining when intrinsification attempt fails?

[jatin-bhateja]

It was to give intrinsic another chance to succeed if it fails due to non-constant context on first attempt,
#24104 (comment)

Currently, if intrinsification fails then we set the generator for CallStaticJavaNode in Compile::inline_incrementally_one
https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/compile.cpp#L2108

Compile::inline_incrementally_cleanup called after Compile::inline_incrementally_one internally calls IGVN optimizations
https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/compile.cpp#L2213

CallStaticJavaNode idealization then re-injects the the failed intrinsic call node to _late_inlines list for another intrinsification attempt.
https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/callnode.cpp#L1175

If we inline the fallback on first intrinsification failure then we loose another opportunity to intrinsify, _vector_late_inlines collects such callgenerators and then once we are through with intrinsification attempts it inline the failed intrinsic calls towards the end on the lines of _string_late_inlines.

https://github.com/openjdk/jdk/pull/24104/changes#diff-f076857d7da81f56709da3de1511b1105727032186cde4d02c678667761f46eaR2252

[iwanowww]

Ok, but you could delay vector operation intrinsification until a full round of late inlining is over and then dispatch between intrinsic and fallback implementation.

Overall, I'm not fully satisfied with current implementation. Please, extract it in a separate PR and let's discuss it there.

[jatin-bhateja]

Hi @iwanowww
This pull request performs partial intrinsification of slice API and if it does not succeed then we attempt inlining vector API based fallback implementation. moving compiler side change into a new PR will also involve factoring out Java side changes related to slice.

I agree with you that existing handling in CallGenerator::do_late_inline_helper is somewhat messy, I have cleaned up the handling for populating _vector_late_lines in the latest patch. Request your to kindly have a re-look at the change and let me know if this looks fine now.

Best Regards

[openjdk]

The total number of required reviews for this PR has been set to 2 based on the presence of these labels: hotspot, hotspot-compiler. This can be overridden with the /reviewers command.