MiningForDudes

Ask HN: How are you managing LLM inference at the edge?

I’m building a system to run small LLMs on-device (mobile, IoT, on-prem servers) and would love to hear how others have tackled the challenges.Context:Use cases: offline chatbots, smart cameras, local data privacyModels: 7–13B parameter quantized models (e.g. Llama 2, Vicuna)Constraints: limited RAM/flash, CPU-only or tiny GPU, intermittent connectivityQuestions:What runtimes or frameworks are you using (ONNX Runtime, TVM, custom C++)?How do you handle model loading, eviction, and batching

Bitcoin has no viable way to implement quantum resistance, yet is near 100k$

Is this the most mispriced asset in the world?

AI decentralized apps are coming for the Web3 throne: DappRadar

AI decentralized apps (DApps) have seen a spike in user activity and could soon challenge gaming and DeFi for the top spot in the DApp ecosystem, according to blockchain analytics platform DappRadar.Gaming and DeFi are both sitting on 21% dominance in April, judged by percentage of unique active wallets, while AI has climbed to 16%, up from the 11% recorded in the February report, data in DappRadar’s April industry report shows.“As user interest in artificial intelligence tools grows

Apple makes progress toward its first pair of smart glasses: Report

Apple is reportedly working on its own microchips across multiple product categories, including smart glasses and artificial intelligence — a hint at what’s next for the massive Silicon Valley-based tech giant. A May 8 report from Bloomberg, citing people familiar with the matter, said the company is working on new processors to power its future devices, including its first smart glasses to rival Meta’s Ray-Bans, more powerful Macs, and artificial intelligence servers.The smart

Zerebro dev is reportedly alive and at parents’ house: SF Standard

The 22-year-old developer of Zerebro, who apparently committed suicide during a livestream on May 4, is actually alive, according to a San Francisco news outlet that claims they spoke to Yu outside his family home. The San Francisco Standard reporter George Kelly claimed on May 8 that he briefly spoke with Yu outside of his family’s two-story home, where the crypto influencer refused to discuss the suicide allegations and whether he had financially benefited from it. Instead, Yu

US man who sent crypto to ISIS could serve prison till he’s 65

A man from the US state of Virginia will spend over three decades behind bars after being convicted of sending crypto to the terrorist organization commonly known as the Islamic State of Iraq and Syria.Federal Judge David Novak sentenced Mohammed Azharuddin Chhipa to 30 years and four months in prison on May 7 for sending over $185,000 to the Islamic State, the Department of Justice said on May 8.Prosecutors said that from around October 2019 until October 2022, the 35-year-old Chhipa collected

Rumble CEO confirms Tether-collab crypto wallet to launch in Q3

Rumble’s pro-crypto founder and CEO has confirmed that the firm will launch its Bitcoin and stablecoin wallet in the third quarter of this year, aimed at giving the Coinbase Wallet a run for its money. The Rumble Wallet will be launched in partnership with stablecoin issuer Tether and compete directly with Coinbase, Chris Pavlovski said in a May 9 X post.“Our goal is to become the most prominent non-custodial Bitcoin and stablecoin wallet, powering the creator economy,” he

Bitcoin at $103K hurtles MARA stack toward $5B, holdings triple

Bitcoin mining firm MARA Holdings (MARA) nearly tripled its Bitcoin holdings over 12 months, according to its newly released Q1 results.However, its Bitcoin production fell, and total earnings slightly missed Wall Street estimates in Q1.MARA, formerly Marathon Digital, saw its Bitcoin (BTC) holdings increase to 47,531 BTC, up 175% from the 17,320 BTC the firm was holding at the end of Q1 2024.MARA holdings inch closer to $5B after Bitcoin pumpMARA holds the second-largest amount of Bitcoin among

SEC’s Crenshaw slams Ripple settlement, warns of ‘regulatory vacuum’

A crypto-skeptical commissioner at the US Securities and Exchange Commission has blasted her agency over its settlement letter that could finally end the Ripple legal saga. The SEC and Ripple filed a joint settlement letter in a New York court asking for the August 2024 injunction against Ripple to be dissolved and $75 million of the $125 million in civil penalties held in escrow to be returned to the crypto firm, according to a May 8 statement from the SEC.SEC Commissioner Caroline Crenshaw bla

Coinbase revenue falls 10% in Q1, missing industry estimate

Crypto exchange Coinbase’s total revenue fell 10% quarter-over-quarter to $2 billion in Q1, missing industry estimates by 4.1% as trading activity slowed across the market.Coinbase’s net income was sliced by 95% from a near-company record $1.29 billion in Q4 to $66 million, in a large part due to Coinbase marking a $596 million paper loss on its crypto holdings. The firm’s earnings per share of $1.94, however, managed to beat the Zacks Consensus Estimate of $1.85 for the quarte

Meta exploring stablecoin integration for payouts: Report

Tech company Meta is reportedly exploring integrating stablecoin payments into its platforms after a three-year hiatus from cryptocurrencies, Fortune reported, citing sources familiar with the matter.The Facebook parent held talks with several crypto infrastructure firms in consultation but has not chosen a decisive course of action, according to the report.One source said the company may take a multi-token approach and integrate support for popular stablecoins such as Tether's USDt (USDT), Circ

Wellgistics Health to integrate XRP into payment infrastructure

Wellgistics Health, a healthcare infrastructure company, will integrate XRP (XRP) and related technologies into its payment network to streamline transactions between pharmacies, medical suppliers and prescription medication manufacturers, the company said in an announcement on May 8. Wellgistics cited the finality time of XRP transactions and reduced transaction costs, which are fractions of a penny, compared to legacy financial architecture like automated clearinghouse (ACH) payments or wire t

SEC considers new rules easing security token issuance

The US Securities and Exchange Commission (SEC) is considering rule changes to let companies more freely issue tokenized securities, SEC Commissioner Hester Peirce said in a speech published on May 8.The regulator is “considering a potential exemptive order” for firms using blockchain technology to “issue, trade, and settle securities” that would release them from certain registration requirements, Peirce said in the speech.For example, decentralized exchanges (DEXs) may

Are layer 2s good for Ethereum, or are they ‘extractive?’

Layer 2s have been a great blockchain success story. They’ve reduced congestion on the Ethereum mainnet, driving down gas fees while preserving security.But maybe they’ve become too successful, drawing chain activity and fee income from the parent that spawned them? At least that’s what some are suggesting lately, most recently at Cornell Tech’s blockchain conference in late April.Indeed, some think Ethereum should be a little greedier, or at least fight harder for a bigg

Missouri bill ending capital gains tax heads to governor for signature

Missouri House Bill 594, a bill that would eliminate capital gains tax in the US state, has passed a vote in the state House of Representatives and now heads to Missouri Governor Mike Kehoe's desk for signature.According to attorney Aaron Brogan, the bill stipulates a 100% income tax deduction for any capital gains income because the Missouri tax code does not explicitly distinguish between capital gains and income tax.Missouri House Bill 594 proposes exempting capital gains from income taxes. S

More ways to pay? ChatGPT may be exploring weekly and lifetime subscriptions (APK teardown)

ChatGPT may soon introduce weekly and lifetime subscriptions alongside monthly and annual plans, likely for ChatGPT Plus. Read on!

AT&T's switch from ChatGPT to open-source AI helped it hang on to thousands of customers

AT&T's switch to open-source AI from ChatGPT enabled cost savings and efficiency, helping the company retain thousands of ...

ChatGPT’s Largest Enterprise User, PwC, Cuts Workforce By 1,500 Jobs

ChatGPT's largest enterprise customer, PwC, cut about 2% of its U.S. workforce this week. Will other Big 4 services firms follow suit?

ChatGPT has unraveled the entire academic project

“After spending the better part of the past two years grading AI-generated papers, Troy Jollimore, a poet, philosopher, and ...

ChatGPT Does Not Want You Cheating on It With Claude

we have greatly improved memory in chatgpt--it can now reference all your past conversations! this is a surprisingly great ...