Show HN: Route your prompts to the best LLM https://ift.tt/l26p35t

Show HN: Route your prompts to the best LLM Hey HN, we've just finished building a dynamic router for LLMs, which takes each prompt and sends it to the most appropriate model and provider. We'd love to know what you think! Here is a quick(ish) screen-recroding explaining how it works: https://youtu.be/ZpY6SIkBosE Best results when training a custom router on your own prompt data: https://youtu.be/9JYqNbIEac0 The router balances user preferences for quality, speed and cost. The end result is higher quality and faster LLM responses at lower cost. The quality for each candidate LLM is predicted ahead of time using a neural scoring function, which is a BERT-like architecture conditioned on the prompt and a latent representation of the LLM being scored. The different LLMs are queried across the batch dimension, with the neural scoring architecture taking a single latent representation of the LLM as input per forward pass. This makes the scoring function very modular to query for different LLM combinations. It is trained in a supervised manner on several open LLM datasets, using GPT4 as a judge. The cost and speed data is taken from our live benchmarks, updated every few hours across all continents. The final "loss function" is a linear combination of quality, cost, inter-token-latency and time-to-first-token, with the user effectively scaling the weighting factors of this linear combination. Smaller LLMs are often good enough for simple prompts, but knowing exactly how and when they might break is difficult. Simple perturbations of the phrasing can cause smaller LLMs to fail catastrophically, making them hard to rely on. For example, Gemma-7B converts numbers to strings and returns the "largest" string when asking for the "largest" number in a set, but works fine when asking for the "highest" or "maximum". The router is able to learn these quirky distributions, and ensure that the smaller, cheaper and faster LLMs are only used when there is high confidence that they will get the answer correct. Pricing-wise, we charge the same rates as the backend providers we route to, without taking any margins. We also give $50 in free credits to all new signups. The router can be used off-the-shelf, or it can be trained directly on your own data for improved performance. What do people think? Could this be useful? Feedback of all kinds is welcome! https://ift.tt/c07Iov8 May 22, 2024 at 08:37PM

Show HN: B-field, a novel probabilistic key-value data structure (`rust-bfield`) https://ift.tt/vWqU4CV

Show HN: B-field, a novel probabilistic key-value data structure (`rust-bfield`) `rust-bfield` is a Rust implementation of our novel "B-field" data structure, which functions like a Bloom filter for key-value lookups instead of set membership queries. The B-field allows you to compactly store data using only a few bytes per key-value pair. We've successfully utilized it in genomics to associate billions of "k-mers" with taxonomic identifiers while maintaining an efficient memory footprint. But the data structure is also useful beyond computational biology, particularly where you have large unique key domains and constrained value ranges. Available under an Apache 2 license. We hope it proves useful, and we're happy to answer any questions! https://ift.tt/8SePW0N May 22, 2024 at 11:23PM

Show HN: Adblock for Podcasts https://ift.tt/a1wUKPp

Show HN: Adblock for Podcasts This is a small app that achieves surprisingly good podcast adblocking. It transcribes the podcast, identifies ad segments in the transcript, then creates a new version of the podcast without the ads. https://ift.tt/fapLz1h May 22, 2024 at 05:31AM

Show HN: An online billboard I built in 2 weeks https://ift.tt/JtWMOdB

Show HN: An online billboard I built in 2 weeks Never built something this quick from the ground up! https://ift.tt/TgIMz5Z May 22, 2024 at 01:19AM

Show HN: I recreated Jeffery Ventrella's "Particle Life" https://ift.tt/LHkFNJ3

Show HN: I recreated Jeffery Ventrella's "Particle Life" This weekend's coding session was dedicated to recreating Jeffery Ventrella's "Particle Life". As you can imagine, I had a lot of fun with it. I recently saw this here on Hacker News and found it strange that I’d never come across it before. Source code: https://ift.tt/due2GV3 The color palette is by Alex Cristache. https://twitter.com/AlexCristache/status/1791122643816431930 https://ift.tt/v2ehfCV May 18, 2024 at 01:47AM

Show HN: Optigraph – optimum graph network generator https://ift.tt/jQlnT4y

Show HN: Optigraph – optimum graph network generator I've created a tool that helps plan graph networks for the best possible connections between nodes. The idea is for it to be used as a kind of underground system planner. I am still working on improving the algorithms it uses, but please consider checking it out for new ideas/bug catching. https://ift.tt/kgVHd17 May 19, 2024 at 03:00PM

Show HN: SQLFrame – I ran PySpark without Spark on a SQL database https://ift.tt/bDHr6M5

Show HN: SQLFrame – I ran PySpark without Spark on a SQL database Recently I open-sourced SQLFrame, a DataFrame library that implements the PySpark DataFrame API but removes Spark as a dependency. It does this by generating the corresponding SQL for the DataFrame operations using SQLGlot. Since the output is SQL this also means that the PySpark DataFrame API can now be used directly against other databases without the Spark middleman. I built this because of two common problems I have faced in my career: 1. I prefer to write complex pipelines in PySpark but they can be hard to read for SQL-proficient co-workers. Therefore I find myself in a tradeoff between maintainability and accessibility. 2. I really enjoy using the PySpark DataFrame API but not every project requires Spark and therefore I'm not able to use the DataFrame library I am most proficient in. The library currently focuses on transformation pipelines (reading from and writing to tables) and data analysis as key use cases. It does offer some ability to read from files directly but they must be small although this can be improved over time if there is demand for it. SQLFrame currently supports BigQuery, DuckDB, and Postgres with Clickhouse, Redshift, Snowflake, Spark, and Trino in development or planned. You can use the "Standalone" session to test running against any engine supported by SQLGlot but there could be issues with more advanced functions that will be resolved once officially supported by SQLFrame. Blog post for more details: https://ift.tt/H6hnLtG... Would love to answer any questions or hear any feedback you may have! https://ift.tt/nW5XYBH May 21, 2024 at 06:39AM

Show HN: Recall Memory – Your Personal Local Time Machine for Mac https://ift.tt/uOVi7J0

Show HN: Recall Memory – Your Personal Local Time Machine for Mac Recall Memory is a free Sandboxed Mac App which let's you scroll through time and allows you to "Recall" what you were doing earlier. Recall works by capturing the active window every second and only saves screenshots with significant changes. Making it easy to scroll through what you were doing earlier. All data is processed and always stored locally. https://ift.tt/R1doNTG May 21, 2024 at 05:18AM

Show HN: Schedule SSH Execution of Bash/PowerShell/SQL Scripts https://ift.tt/ptKQrJx

Show HN: Schedule SSH Execution of Bash/PowerShell/SQL Scripts https://ctfreak.com May 21, 2024 at 02:04AM

Show HN: DeLorean – Super Performant IPv4->IPv6 Reverse Proxy in Golang https://ift.tt/iZGlHWV

Show HN: DeLorean – Super Performant IPv4->IPv6 Reverse Proxy in Golang https://ift.tt/iGsgAnU May 20, 2024 at 08:25AM

Show HN: ffmpeg-english "capture from /dev/video0 every 1 second to jpg files" https://ift.tt/y0pI5e6

Show HN: ffmpeg-english "capture from /dev/video0 every 1 second to jpg files" https://ift.tt/M8Jkfup May 20, 2024 at 05:20AM

Show HN: A VS Code extension to check incompatible CSS https://ift.tt/O4XUPKW

Show HN: A VS Code extension to check incompatible CSS I've developed a handy tool for Visual Studio Code that makes checking CSS compatibility very easy for developers. It provides instant feedback on syntax, keywords, types, and function compatibility. It even highlights deprecated, non-standard, and experimental features. https://ift.tt/whrRsBt May 19, 2024 at 11:24PM

Show HN: I Built a Tool to Stop Cold Email Spam https://ift.tt/Fu4ZEyO

Show HN: I Built a Tool to Stop Cold Email Spam Hi HN! I'm Louw, founder of the enterprise insurtech scale-up Root[0]. Over the last 8 years, my inbox has been flooded with unsolicited emails, especially since early 2023. Recruitment services, outsourced software development agencies, and a crazy amount of lead generation services (thanks, Apollo!) drowned my inbox. To solve this, I created Unspam. Initially, it was a simple tool for myself and fellow founders to reduce noise. And it worked — about 10% of my emails were swiftly and accurately categorized as unsolicited messages. Given its success, I've decided to package and share Unspam with the broader community, evolving it into Unspam.io. How it works: 1. Hook up your Gmail 2. Unspam monitors received mail in the background 3. Unsolicited emails are labeled and archived Please try it out and let me know your feedback. What would you improve? https://unspam.io --- [0] https://ift.tt/xNEgvCf [1] PS: I value privacy intensely. Unspam doesn’t record, store, or train on your emails. No need to. https://unspam.io May 19, 2024 at 12:30AM

Show HN: A Golang CP/M emulator https://ift.tt/8xOI7sz

Show HN: A Golang CP/M emulator https://ift.tt/k86LS5F May 18, 2024 at 11:18PM

Show HN: I built an analytics tool to track cost and usage of any LLM provider https://ift.tt/bHp395K

Show HN: I built an analytics tool to track cost and usage of any LLM provider https://viewpointhq.com May 18, 2024 at 02:38AM

Show HN: Drivr – VR with real vehicles [video] https://ift.tt/sKc1pza

Show HN: Drivr – VR with real vehicles [video] This project combines VR with a real vehicle that you are controlling (or is controlling itself, in some scenarios). One advantage this has over traditional VR motion experiences is that your senses all agree with each other, thus greatly reducing the likelihood of motion sickness. The go-kart seen in this video is drive-by-wire and steer-by-wire, so the system will not let the player exit the defined safe area. If a player goes off course, the vehicle will take control, and return them to the start. The autonomous capabilities also allow for games where the player can focus on other objectives other than driving, such as target shooting. No infrastructure is required other than the vehicle and an open place to drive (which honestly has been the most challenging part lately). If you're interested in this project, I'd love to connect! https://www.youtube.com/watch?v=Q76X0cnSGHk May 17, 2024 at 12:49AM

Show HN: Open-Source RAG Evaluation Toolkit https://ift.tt/haP6dxC

Show HN: Open-Source RAG Evaluation Toolkit https://ift.tt/n1P7RdA May 17, 2024 at 09:18PM

Show HN: Syntax highlighting tool for code snippets in HTML https://ift.tt/0Ze9FaT

Show HN: Syntax highlighting tool for code snippets in HTML Hey! My name is Vojtěch. When I'm sharing code on my blog or elsewhere, I find it really annoying to format and highlight all the words in the code snippet. That's why I made CodeGlow, to make sharing sharing code easy and painless. https://ift.tt/5GrIQKS May 16, 2024 at 08:48PM

Show HN: I made an AI-based domain name generator with availability check https://ift.tt/GgiW3yh

Show HN: I made an AI-based domain name generator with availability check This project came from my own personal need. There are many AI based domain name generators on the market but most of them don't check for availability. Hopefully it helps someone else. https://ift.tt/aNdOREj May 16, 2024 at 10:02PM

Show HN: Smart website search powered by open models https://ift.tt/6pVcqCg

Show HN: Smart website search powered by open models https://ift.tt/IHF182w May 16, 2024 at 12:33AM