artin/llama2.c
1
0
Fork 0
mirror of https://github.com/trholding/llama2.c.git synced 2026-02-06 11:26:53 +00:00
Code Issues Packages Projects Releases Wiki Activity
1c47da5ebf
llama2.c/llama3_export
History
Vulcan 5d981db385 Add llama3 tokenizer export script and tokenizer binary 
The llama3 tokenizer export script is taken from @jameswdelancey 's llama3.c project:

https://github.com/jameswdelancey/llama3.c

Much appreciation & credits to  @jameswdelancey
2024-07-10 10:38:20 +05:30
..
requirements.txt Add llama3 tokenizer export script and tokenizer binary 2024-07-10 10:38:20 +05:30
tokenizer.py Add llama3 tokenizer export script and tokenizer binary 2024-07-10 10:38:20 +05:30