Save to wishlistSave to wishlist
Ingest-time enrichment. Rather than relying solely on runtime search, offline compute at ingest time (entity extraction, relationship mapping, summary generation) could provide the agent with a richer substrate to search over. This trades offline compute for runtime efficiency, a tradeoff that becomes increasingly favorable as models get faster and the cost of inference continues to decrease.
，详情可参考金山文档
Go to worldnews
Henry：对。谷歌的TCO好的原因，就是有海量客户同时去用这样一个推理服务，它的吞吐量就会很高，但它不在乎尾部延迟。尾部延迟指的就是单用户用的话，它可能会有时候会快，有时候会慢一点，相信大家也都会有这样的体验。但是Groq，你一旦去用的话，它就会非常非常快。它的原理第一是它的SRAM静态随机存取存储器。第二，它是一个相当于你一个人占用了非常多的LPU资源，而不是跟很多人去共享。