Benchmarking DIY LLM on Cheap Tablet

Hi Everybody! I just wanted to share some progress that I have been making on BULaMU, the world's first large language model that has been trained from scratch on Luganda. I built a small android app to see how the 20M parameter version of BULaMU would perform on low-cost devices, like the 2021 Amazon Fire HD 10, which has 3GB of RAM. The 20M parameter model was able to get 4.7-4.8 tokens a second on my Fire Tablet when inferencing using Kotlin. submitted by /u/AgencyInside407 [link] [comments.