Skip to content

LCORE-1728: Upstream transformers #1756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tisnik wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

LCORE-1728: Upstream transformers #1756

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d7c4987
Upstream transformers
tisnik May 18, 2026
e3a7a86
Upstream transformers
tisnik May 18, 2026
f8f8cc6
hf-xet
tisnik May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions .konflux/requirements.hashes.source.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -491,9 +491,35 @@ greenlet==3.3.2 \
grpc-google-iam-v1==0.14.4 \
--hash=sha256:392b3796947ed6334e61171d9ab06bf7eb357f554e5fc7556ad7aab6d0e17038 \
--hash=sha256:412facc320fcbd94034b4df3d557662051d4d8adfa86e0ddb4dca70a3f739964
huggingface-hub==0.36.2 \
--hash=sha256:1934304d2fb224f8afa3b87007d58501acfda9215b334eed53072dd5e815ff7a \
--hash=sha256:48f0c8eac16145dfce371e9d2d7772854a4f591bcb56c9cf548accf531d54270
hf-xet==1.5.0 \
--hash=sha256:1e60df5a42e9bed8628b6416af2cba4cba57ae9f02de226a06b020d98e1aab18 \
--hash=sha256:2806c7c17b4d23f8d88f7c4814f838c3b6150773fe339c20af23e1cfaf2797e4 \
--hash=sha256:2baea1b0b989e5c152fe81425f7745ddc8901280ba3d97c98d8cdece7b706c60 \
--hash=sha256:3531b1823a0e6d77d80f9ed15ca0e00f0d115094f8ac033d5cae88f4564cc949 \
--hash=sha256:4b35549ce62601b84da4ff9b24d970032ace3d4430f52d91bcbb26c901d6c690 \
--hash=sha256:526345b3ed45f374f6317349df489167606736c876241ba984105afe7fd4839d \
--hash=sha256:5906bf7718d3636dc13402914736abe723492cb730f744834f5f5b67d3a12702 \
--hash=sha256:5f3dc2248fc01cc0a00cd392ab497f1ca373fcbc7e3f2da1f452480b384e839e \
--hash=sha256:73a0dae8c71de3b0633a45c73f4a4a5ed09e94b43441d82981a781d4f12baa42 \
--hash=sha256:786d28e2eb8315d5035544b9d137b4a842d600c434bb91bf7d0d953cce906ad4 \
--hash=sha256:7d70fe2ce97b9db73b9c9b9c81fe3693640aec83416a966c446afea54acfae3c \
--hash=sha256:872d5601e6deea30d15865ede55d29eac6daf5a534ab417b99b6ef6b076dd96c \
--hash=sha256:8dbcbab554c9ef158ef2c991545c3e970ddd8cc7acdcd0a78c5a41095dab4ded \
--hash=sha256:9929561f5abf4581c8ea79587881dfef6b8abb2a0d8a51915936fc2a614f4e73 \
--hash=sha256:9a0ee58cd18d5ea799f7ed11290bbccbe56bdd8b1d97ca74b9cc49a3945d7a3b \
--hash=sha256:a60290ec57e9b71767fba7c3645ddafdd0759974b540441510c629c6db6db24a \
--hash=sha256:b285cea1b5bab46b758772716ba8d6854a1a0310fed1c249d678a8b38601e5a0 \
--hash=sha256:b6c9df403040248c76d808d3e047d64db2d923bae593eb244c41e425cf6cd7be \
--hash=sha256:c799d49f1a5544a0ef7591c0ee75e0d6b93d6f56dc7a4979f59f7518d2872216 \
--hash=sha256:cf7b2dc6f31a4ea754bb50f74cde482dcf5d366d184076d8530b9872787f3761 \
--hash=sha256:dad0dc84e941b8ba3c860659fe1fdc35c049d47cce293f003287757e971a8f56 \
--hash=sha256:e0fb0a34d9f406eed88233e829a67ec016bec5af19e480eac65a233ea289a948 \
--hash=sha256:e5de0f6deada0dada870bb376a11bcd1f08abf3a968a6d118f33e72d1b1eb480 \
--hash=sha256:f7b7bbae318e583a86fb21e5a4a175d6721d628a2874f4bd022d0e660c32a682 \
--hash=sha256:fd6e5a9b0fdac4ed03ed45ef79254a655b1aaab514a02202617fbf643f5fdf7a
huggingface-hub==1.15.0 \
--hash=sha256:28abfdddda3927fd4de6a63cf26ab012498a2c24dae52baf150c5c6edf98a1d5 \
--hash=sha256:a4a59af04cbc41a3fe3fec429b171ef994ef8c971eda10136746f408dd4e3744
importlib-metadata==8.5.0 \
--hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \
--hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7
Expand Down Expand Up @@ -1042,6 +1068,9 @@ tornado==6.5.5 \
--hash=sha256:65a7f1d46d4bb41df1ac99f5fcb685fb25c7e61613742d5108b010975a9a6521 \
--hash=sha256:dd3eafaaeec1c7f2f8fdcd5f964e8907ad788fe8a5a32c4426fbbdda621223b7 \
--hash=sha256:e74c92e8e65086b338fd56333fb9a68b9f6f2fe7ad532645a290a464bcf46be5
transformers==5.8.1 \
--hash=sha256:4dd5b6de4105725104d84fd6abd74b305f4debfc251b38c648ee5dd087cf543b \
--hash=sha256:5340fb95962162cdfdae5cc91d7f8fedd92ed75216c1154c5e1f590fcf56dd0e
trl==1.0.0 \
--hash=sha256:529f447b77007c8d181a3cf3d8b1b6427888cac9ee1e6d583bd0dbda78165ed4 \
--hash=sha256:c45700cb6aa8723e5db1dd8773297f642715accab1ca7754b56e6d50a37dfb1d
Expand Down
5 changes: 0 additions & 5 deletions .konflux/requirements.hashes.wheel.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,6 @@ grpcio-status==1.76.0 \
--hash=sha256:257105544721149a1fabeb0465dc622add2a7b6f1be07df4ea9ddcb741113cfa
h11==0.16.0 \
--hash=sha256:a485f5bfa77401603f78cff4db5bb4dbaadb947a3da8eae068cd05b6bb4aab66
hf-xet==1.2.0 \
--hash=sha256:a14cc5078bf43efc85ad180ec02372e5a84d134de095c0a9740f7e0407df7866 \
--hash=sha256:acb8669cac4d9aa8e8ad31a92d99353fe2f0d4463b8aaa54f2b93a33e222149d
httpcore==1.0.9 \
--hash=sha256:5af3f1b09e795d5548d5030be3d4191e65113cfdac21968c82bf0cddf15ec313
httpx==0.28.1 \
Expand Down Expand Up @@ -219,8 +216,6 @@ torch==2.9.1 \
--hash=sha256:e42ab849b64444059f5eda352d61c9c3a078f30797e48e2972857182c9a00cf8
tqdm==4.67.3 \
--hash=sha256:d798b33fcc041b9a42c57f462b9c068a5a15c2dbcef1c87695d80c7074770a4d
transformers==4.57.6 \
--hash=sha256:7c35d073118fad72b1745bd3dad3149d9bf2cacca3b16a7ed8f47f1a0bec119d
tree-sitter==0.25.2 \
--hash=sha256:05bc490c99a846e0e91d9dd0c5067de48b35b46d4c3f18fc2cc9ac08a4ec71fe \
--hash=sha256:3aa0cf6b0964495ada302c3eb966bf6f41a01279d8e31c019e4b0129a93d5a55 \
Expand Down
2 changes: 1 addition & 1 deletion .konflux/requirements.overrides.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# override these package to the version available on RHOAI wheels index:
# https://console.redhat.com/api/pypi/public-rhai/rhoai/3.3/cpu-ubi9/simple
transformers==4.57.6
# transformers==4.57.6
tokenizers==0.22.2
scipy==1.17.0
# aiohttp==3.13.3
Expand Down
2 changes: 1 addition & 1 deletion .tekton/lightspeed-stack-pull-request.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ spec:
],
"requirements_build_files": ["requirements-build.txt"],
"binary": {
"packages": "aiohappyeyeballs,aiosignal,aiosqlite,annotated-doc,annotated-types,anyio,asyncpg,cffi,chevron,cryptography,click,dill,distro,dnspython,docstring-parser,durationpy,einops,email-validator,faiss-cpu,fire,frozenlist,fsspec,google-crc32c,google-genai,grpcio,grpcio-status,h11,hf-xet,httpcore,httpx,httpx-sse,idna,importlib-metadata,jinja2,jiter,joblib,jsonschema-specifications,kubernetes,markdown-it-py,mdurl,mpmath,multidict,networkx,numpy,oauthlib,packaging,pandas,peft,pillow,prometheus-client,prompt-toolkit,propcache,psycopg2-binary,pyarrow,pyasn1-modules,pycparser,pydantic,pydantic-core,python-dateutil,pyyaml,referencing,requests-oauthlib,rpds-py,safetensors,scikit-learn,scipy,setuptools,six,sniffio,sqlalchemy,sympy,termcolor,threadpoolctl,tiktoken,tokenizers,torch,tqdm,transformers,tree-sitter,triton,typing-extensions,typing-inspection,websocket-client,websockets,wrapt,xxhash,yarl,zipp,uv,pip,maturin",
"packages": "aiohappyeyeballs,aiosignal,aiosqlite,annotated-doc,annotated-types,anyio,asyncpg,cffi,chevron,cryptography,click,dill,distro,dnspython,docstring-parser,durationpy,einops,email-validator,faiss-cpu,fire,frozenlist,fsspec,google-crc32c,google-genai,grpcio,grpcio-status,h11,httpcore,httpx,httpx-sse,idna,importlib-metadata,jinja2,jiter,joblib,jsonschema-specifications,kubernetes,markdown-it-py,mdurl,mpmath,multidict,networkx,numpy,oauthlib,packaging,pandas,peft,pillow,prometheus-client,prompt-toolkit,propcache,psycopg2-binary,pyarrow,pyasn1-modules,pycparser,pydantic,pydantic-core,python-dateutil,pyyaml,referencing,requests-oauthlib,rpds-py,safetensors,scikit-learn,scipy,setuptools,six,sniffio,sqlalchemy,sympy,termcolor,threadpoolctl,tiktoken,tokenizers,torch,tqdm,tree-sitter,triton,typing-extensions,typing-inspection,websocket-client,websockets,wrapt,xxhash,yarl,zipp,uv,pip,maturin",
"os": "linux",
"arch": "x86_64,aarch64",
"py_version": 312
Expand Down
2 changes: 1 addition & 1 deletion .tekton/lightspeed-stack-push.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ spec:
],
"requirements_build_files": ["requirements-build.txt"],
"binary": {
"packages": "aiohappyeyeballs,aiosignal,aiosqlite,annotated-doc,annotated-types,anyio,asyncpg,cffi,chevron,cryptography,click,dill,distro,dnspython,docstring-parser,durationpy,einops,email-validator,faiss-cpu,fire,frozenlist,fsspec,google-crc32c,google-genai,grpcio,grpcio-status,h11,hf-xet,httpcore,httpx,httpx-sse,idna,importlib-metadata,jinja2,jiter,joblib,jsonschema-specifications,kubernetes,markdown-it-py,mdurl,mpmath,multidict,networkx,numpy,oauthlib,packaging,pandas,peft,pillow,prometheus-client,prompt-toolkit,propcache,psycopg2-binary,pyarrow,pyasn1-modules,pycparser,pydantic,pydantic-core,python-dateutil,pyyaml,referencing,requests-oauthlib,rpds-py,safetensors,scikit-learn,scipy,setuptools,six,sniffio,sqlalchemy,sympy,termcolor,threadpoolctl,tiktoken,tokenizers,torch,tqdm,transformers,tree-sitter,triton,typing-extensions,typing-inspection,websocket-client,websockets,wrapt,xxhash,yarl,zipp,uv,pip,maturin",
"packages": "aiohappyeyeballs,aiosignal,aiosqlite,annotated-doc,annotated-types,anyio,asyncpg,cffi,chevron,cryptography,click,dill,distro,dnspython,docstring-parser,durationpy,einops,email-validator,faiss-cpu,fire,frozenlist,fsspec,google-crc32c,google-genai,grpcio,grpcio-status,h11,httpcore,httpx,httpx-sse,idna,importlib-metadata,jinja2,jiter,joblib,jsonschema-specifications,kubernetes,markdown-it-py,mdurl,mpmath,multidict,networkx,numpy,oauthlib,packaging,pandas,peft,pillow,prometheus-client,prompt-toolkit,propcache,psycopg2-binary,pyarrow,pyasn1-modules,pycparser,pydantic,pydantic-core,python-dateutil,pyyaml,referencing,requests-oauthlib,rpds-py,safetensors,scikit-learn,scipy,setuptools,six,sniffio,sqlalchemy,sympy,termcolor,threadpoolctl,tiktoken,tokenizers,torch,tqdm,tree-sitter,triton,typing-extensions,typing-inspection,websocket-client,websockets,wrapt,xxhash,yarl,zipp,uv,pip,maturin",
"os": "linux",
"arch": "x86_64,aarch64",
"py_version": 312
Expand Down
Loading