Show HN: Tune LLaMa3.1 on Google Cloud TPUs https://ift.tt/Y1rKRxe

Show HN: Tune LLaMa3.1 on Google Cloud TPUs Hey HN, we wanted to share our repo where we fine-tuned Llama 3.1 on Google TPUs. We’re building AI infra to fine-tune and serve LLMs on non-NVIDIA GPUs (TPUs, Trainium, AMD GPUs). The problem: Right now, 90% of LLM workloads run on NVIDIA GPUs, but there are equally powerful and more cost-effective alternatives out there. For example, training and serving Llama 3.1 on Google TPUs is about 30% cheaper than NVIDIA GPUs. But developer tooling for non-NVIDIA chipsets is lacking. We felt this pain ourselves. We initially tried using PyTorch XLA to train Llama 3.1 on TPUs, but it was rough: xla integration with pytorch is clunky, missing libraries (bitsandbytes didn't work), and cryptic HuggingFace errors. We then took a different route and translated Llama 3.1 from PyTorch to JAX. Now, it’s running smoothly on TPUs! We still have challenges ahead, there is no good LoRA library in JAX, but this feels like the right path forward. Here's a demo ( https://ift.tt/XI6x4pf ) of our managed solution. Would love your thoughts on our repo and vision as we keep chugging along! https://ift.tt/hPQfkaG September 11, 2024 at 08:44PM

Show HN: Kolors virtual try on, AI change clothes save time and money https://ift.tt/gGxCywp

Show HN: Kolors virtual try on, AI change clothes save time and money Unlock your perfect dressing style effortlessly with our virtual try on clothes tool. Upload a photo of yourself and enjoy multiple virtual clothing options before buying. Let our AI virtual try on tool save your time and boost your confidence. https://ift.tt/nKoBCtJ September 11, 2024 at 06:49AM

Show HN: I put together boring hex code color to save you time https://ift.tt/c0tZ6Dp

Show HN: I put together boring hex code color to save you time I made this tool to help users get hex codes that I put together based on the seasons I find beautiful. I hope you like it https://ift.tt/nlXyCgK September 11, 2024 at 07:02AM

Show HN: Forms with OpenPGP https://ift.tt/5N4PEkQ

Show HN: Forms with OpenPGP I'm Pal, the creator of an open source form/survey platform. It's written in Rust and Svelte and it's all on GitHub: https://ift.tt/p2RB7V6 . My aim is to do something similar to how Proton encrypted emails: keep it simple and user-friendly, so that basically everyone can switch to it from Google/Typeform. Of course, there's some compromises. You can't really do integrations, since the server doesn't have the raw responses to send anywhere. But there's still tons of use cases where Palform's features suffice. The encryption is powered by the amazing Sequoia PGP library and simply uses OpenPGP. I know it's not a perfect protocol, but it's been around for ages and audited several times. Users need to be able to trust Palform, and a DIY obscure cryptographic system would make that hard. Plus, this way you can import + export your keys and everything stays interoperable. It even has webhooks, so your servers can store PGP keys and you can decrypt the form responses yourself. https://palform.app/ September 11, 2024 at 04:23AM

Show HN: Semantic Bookmark Manager https://ift.tt/QAd4Zzv

Show HN: Semantic Bookmark Manager Hi Hacker News, I would like to share my new side project: the Semantic Bookmark Manager. This web application is designed to help users manage and semantically search for their bookmarks using the RAG technique. Traditional bookmark managers can become quite disorganized and difficult to navigate as they grow. This tool offers a solution by eliminating the need for manual categorization, therefore simplifying the overall user experience. It is open-source under MIT license Thank you for your attention, and I hope you find it useful :) https://ift.tt/MoTy5Qp September 11, 2024 at 04:57AM

Show HN: HypergraphZ – A Hypergraph Implementation in Zig https://ift.tt/p5mbdDW

Show HN: HypergraphZ – A Hypergraph Implementation in Zig https://ift.tt/Byflgrb September 10, 2024 at 11:37AM

Show HN: World's most performant web table (multicore, DOM-based) https://ift.tt/eD9gcBH

Show HN: World's most performant web table (multicore, DOM-based) https://ift.tt/wlCohUn September 10, 2024 at 12:11AM

Show HN: Turn Any ArXiv Paper into a 200-Page Prerequisite Reading Book https://ift.tt/wV4scfv

Show HN: Turn Any ArXiv Paper into a 200-Page Prerequisite Reading Book I created this tool over the weekend because, as someone interested in AI and technology, I find many research papers on arXiv fascinating but often incredibly dense and difficult to understand due to the heavy jargon and technical language. I wanted to make these complex topics more accessible to laypeople like myself by converting the topics described in these papers into full-length books that break down key concepts, making cutting-edge research easier to grasp. You can think of it as generating prerequisite reading materials before being able to read the actual paper. Here are some noteworthy arXiv papers you can test this with: - Attention is All You Need: https://ift.tt/txkcC7X - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: https://ift.tt/mhI7qld - GPT-3: Language Models are Few-Shot Learners: https://ift.tt/RG9Uwc6 - Deep Residual Learning for Image Recognition (ResNet): https://ift.tt/L8yVQpK The tool leverages GPT-4o, Perplexity, and Instructor to analyze and break down the complex concepts within these papers. Keep in mind, it's not built for heavy traffic (current capacity is about 50 books per hour) so if things get busy, it may take a bit longer, but the book will arrive via email eventually! https://ift.tt/3DiIca8 September 9, 2024 at 11:53PM

Show HN: Ki Editor - the multicursor syntactical editor https://ift.tt/Gab8oAc

Show HN: Ki Editor - the multicursor syntactical editor Hi everyone, I have been developing this editor, Ki, for over a year, and have employed it substantially in all kinds of development (including Ki itself) for at least 3 months. I think it is mostly crystallized, thus I'm happy to share it with you today. Its main strength is first-class multi-cursor and structural (syntax) editing, which is a rare combination in the realm of editors (TUI or GUI alike). Hope you'll enjoy it! https://ift.tt/FwXe8KA September 9, 2024 at 06:09AM

Show HN: Use Spectre.Console Terminal Widgets in PowerShell https://ift.tt/qgJABVE

Show HN: Use Spectre.Console Terminal Widgets in PowerShell I like spectre.console and wanted to be able to use it in powershell scripts so I built this module. https://ift.tt/JUr78Rz September 9, 2024 at 01:50AM

Show HN: Dump entire Git repos into a single file for LLM prompts https://ift.tt/pmv8ZO9

Show HN: Dump entire Git repos into a single file for LLM prompts Hey! I wanted to share a tool I've been working on. It's still very early and a work in progress, but I've found it incredibly helpful when working with Claude and OpenAI's models. What it does: I created a Python script that dumps your entire Git repository into a single file. This makes it much easier to use with Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. Key Features: - Respects .gitignore patterns - Generates a tree-like directory structure - Includes file contents for all non-excluded files - Customizable file type filtering Why I find it useful for LLM/RAG: - Full Context: It gives LLMs a complete picture of my project structure and implementation details. - RAG-Ready: The dumped content serves as a great knowledge base for retrieval-augmented generation. - Better Code Suggestions: LLMs seem to understand my project better and provide more accurate suggestions. - Debugging Aid: When I ask for help with bugs, I can provide the full context easily. How to use it: Example: python dump.py /path/to/your/repo output.txt .gitignore py js tsx Again, it's still a work in progress, but I've found it really helpful in my workflow with AI coding assistants (Claude/Openai). I'd love to hear your thoughts, suggestions, or if anyone else finds this useful! https://ift.tt/8sLEQhB P.S. If anyone wants to contribute or has ideas for improvement, I'm all ears! September 9, 2024 at 01:38AM

Show HN: Private, local data gathering tool, to make your Digital Twin https://ift.tt/4m7IyYB

Show HN: Private, local data gathering tool, to make your Digital Twin Imagine giving your grand-grand-grand kids yourself on an SD card. And all of that starts with data. https://ift.tt/8VOGklp September 8, 2024 at 01:47AM

Show HN: PeepDB – open-source CLI tool to quickly view SQL database tables https://ift.tt/JuVtqwP

Show HN: PeepDB – open-source CLI tool to quickly view SQL database tables https://ift.tt/KRwmy7Y September 7, 2024 at 08:15PM

Show HN: Track whether cryptocurrencies are being used and for what https://ift.tt/EBSZfOl

Show HN: Track whether cryptocurrencies are being used and for what Hey HN, I'm Jake, and I created Chainspy, a website that aggregates and visualizes on-chain metrics across multiple blockchains like Bitcoin, Ethereum, and others. It's designed to give you insights beyond just price data. Why Chainspy? - On-Chain Metrics: Track blockchain activity (e.g., transactions, active users) across multiple networks, all in one place. - Advanced Charting: Visualize blockchain data with integrated TradingView charts. Ideal for comparing blockchain performance over time. - Wealth Distribution: See how wealth is distributed across wallets and spot centralization trends. - API Access (Coming Soon): Need blockchain data for your projects? We're working on an API—let me know if you're interested! A Sneak Peek: Wealth Distribution: How is wealth spread across wallets? https://ift.tt/RVwpOis https://ift.tt/3Gu5vig Active Transactions: Which chains are seeing the most activity? https://ift.tt/iz60bSV Active Users (addresses): How many people are using the network? https://ift.tt/eD6FZcA I’m just starting out and would love feedback on features and data you'd like to see. If you have a favorite blockchain or metric you'd want tracked, let me know—I’ll try to add it. Let’s make blockchain data easier for everyone to access. https://ift.tt/lpPUKMC September 7, 2024 at 06:01AM

Show HN: Museum Music – generate period-appropriate playlists from artwork https://ift.tt/yJpgl0m

Show HN: Museum Music – generate period-appropriate playlists from artwork This is Museum Music, an app to help experience art with era-appropriate music - photograph an art piece, and it will create a unique Spotify playlist that complements its era and style. We built this from our own experiences visiting museums and thoughts of enhancing the experience with music. It's built with GPT Vision and the Spotify API. Although the initial focus was on art-inspired music, it works surprisingly well on other, day-to-day photos too! https://ift.tt/gWlbHRB September 7, 2024 at 01:52AM

Show HN: PlaceholderJS – Simple and Lightweight Placeholders https://ift.tt/pVsYtza

Show HN: PlaceholderJS – Simple and Lightweight Placeholders Hey all! My name is Owen, I'm the developer behind PlaceholderJS. Since the demise of placeholder.com, I wanted to be able to use placeholders on my React projects via a package so that way I don't have to worry about these services shutting down again in the future. Additionally, for our non-React developers out there, I created a simple CDN that functions similarly via the PlaceholderJS.com domain. In the future, I'd like to add support for more frameworks, placeholder text, additional customization, etc. I'm looking to improve this and continue to support it and would love your feedback. https://ift.tt/4LQR52k September 6, 2024 at 02:31AM

Show HN: Nomadic – Minimize RAG Hallucinations with 1 Hyperparameter Experiment https://ift.tt/2NEk9zW

Show HN: Nomadic – Minimize RAG Hallucinations with 1 Hyperparameter Experiment Hey HN! Mustafa, Lizzie, and Varun here from NomadicML ( https://nomadicml.com ). We’re excited to show you Nomadic ( https://ift.tt/sjNYO01 ): a platform focused on parameter search to continuously optimize AI systems. Here’s a simple demo notebook where you get the best-performing, statistically significant configurations for your RAG — and improve hallucination metrics by 4X in just 5 minutes — with a single Nomadic experiment: https://ift.tt/GjLSgsr Our lightweight library is now live on PyPI (`pip install nomadic`). Try one of the README examples :) Input your model, define an evaluation metric, specify the dataset, and choose which parameters to test. Nomadic emerged from our frustration with existing HPO (hyperparameter optimization) solutions. We heard over and over that for the sake of deploying fast, folks resort to setting HPs through a single, expensive grid search or better yet, intuition-based “vibes”. From fine-tuning to inference, small tweaks to HPs can have a huge impact on performance. We wanted a tool to make that “drunken wander” systematic, quick, and interpretable. So we started building Nomadic - our goal is to create the best parameter search platform out there for your ML systems to keep your hyperparameters, prompts, and all aspects of your AI system production-grade. We started aggregating top parameter search techniques from popular tools and research (Bayesian Optimizations, cost-frugal flavors). Among us: Built Lyft’s driver earnings platform, automated Snowflake’s just-in-time compute resource allocation, became a finalist for the INFORMS Wagner Prize (top prize in industrial optimization), and developed a fintech fraud screening system for half a million consumers. You might say we love optimization. If you’re building AI agents / applications across LLM safety, fintech, support, or especially compound AI systems (multiple components > monolithic models), and want to deeply understand your ML system’s best levers to boost performance as it scales - get in touch. Nomadic is being actively developed. Up next: Supporting text-to-SQL pipelines (TAG) and a Workspace UI (preview it at https://ift.tt/8oaYQ5b ). We’re eager to hear honest feedback, likes, dislikes, feature requests, you name it. If you’re also a optimization junkie, we’d love for you to join our community here https://ift.tt/tzENfJr September 5, 2024 at 11:44PM

Show HN: DevMuse – App to bond over music and code https://ift.tt/Dm17sPZ

Show HN: DevMuse – App to bond over music and code I've found there's a disconnect devs can feel when they are remote only, and one of the ways people have connected since the beginning of humanity has been through music. So I thought it would be interesting to build off these concepts and build a tool that brings a sense of connection to devs across the world. I don't have it fully working just yet. Still have some kinks to work out. What do you think? https://ift.tt/UKbLAif September 6, 2024 at 01:39AM

Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust https://ift.tt/zSUkMBw

Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust Hey HN, we’re Robert, Din and Temirlan from Laminar ( https://www.lmnr.ai ), an open-source observability and analytics platform for complex LLM apps. It’s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust. How is Laminar different from the swarm of other “LLM observability” platforms? On the observability part, we’re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry ( https://ift.tt/1ynHjf5 ), open-source package for span production. The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track “semantic metrics” (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span. We think it’s a win to separate core app logic from LLM event processing. Most devs don’t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working. Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it’s overkill for simple pipelines, but it’s extremely cool and we’ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network. One thing missing from all LLM observability platforms right now is an adequate search over traces. We’re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it’s gonna be crucial part of our platform going forward. We also support evaluations. We loved the “run everything locally, send results to a server” approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we’re pushing hard on them. Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it’s current shape, Laminar is just few weeks old and developing rapidly, we’d love any feedback or for you to give Laminar a try in your LLM projects! https://ift.tt/VDt4BMh September 5, 2024 at 04:22AM

Show HN: Local Bookmark Storage and Archive https://ift.tt/aS1KwUQ

Show HN: Local Bookmark Storage and Archive I decided I needed a reliable backup of my bookmarks, and an archive of those pages for future reference in case they get taken down. I paid for a managed online service that did this for a while, but I was unsatisfied with the archiving reliability. This is pretty much just a 200 line python script, but it does the trick, and now I don't have to worry about losing a copy of a web page that I want to refer to in the future. https://ift.tt/Ng9pZzy September 5, 2024 at 01:29AM