Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
RunAnywhere open-sourced TrnRT, an inference engine for AWS Trainium that enables high-speed processing of GLM-5.3 flash.
Key points
- •It supports inference of GLM-5.3 flash (FP8) on AWS trn2.48xlarge instances.
- •It serves two instances on a single box using a cache-aware router.
- •It can process 3.57M tokens per minute on SemiAnalysis's AgentX benchmarks.