You can use the inference API and our instances (use our hosted load balancer to keep the same base URL for every instance) with a self hosted Open Webui already. You can run it on your local machine in docker. Quite a few people do this already.

📢 Gate Square #Creator Campaign Phase 2# is officially live!
Join the ZKWASM event series, share your insights, and win a share of 4,000 $ZKWASM!
As a pioneer in zk-based public chains, ZKWASM is now being prominently promoted on the Gate platform!
Three major campaigns are launching simultaneously: Launchpool subscription, CandyDrop airdrop, and Alpha exclusive trading — don’t miss out!
🎨 Campaign 1: Post on Gate Square and win content rewards
📅 Time: July 25, 22:00 – July 29, 22:00 (UTC+8)
📌 How to participate:
Post original content (at least 100 words) on Gate Square related to

📢 Gate Square #MBG Posting Challenge# is Live— Post for MBG Rewards!
Want a share of 1,000 MBG? Get involved now—show your insights and real participation to become an MBG promoter!
💰 20 top posts will each win 50 MBG!
How to Participate:
1️⃣ Research the MBG project
Share your in-depth views on MBG’s fundamentals, community governance, development goals, and tokenomics, etc.
2️⃣ Join and share your real experience
Take part in MBG activities (CandyDrop, Launchpool, or spot trading), and post your screenshots, earnings, or step-by-step tutorials. Content can include profits, beginner-friendl

🎉 Gate Square’s "Spark Program" Surpasses 1,000 KOLs!
💥 The creator ecosystem is in full bloom!
📈 Get featured, earn rewards, and grow your influence—what are you waiting for?
💰 Cash incentives ✔️
🚀 Traffic support ✔️
👑 Exclusive verification ✔️
From 0 to 1,000 in just weeks—Gate Square is becoming the epicenter of Web3 content! ⚡
You’re not just posting content, but the next "viral opportunity"!
🌟 Join the Spark Program and kickstart your breakthrough!
👉 https://www.gate.com/announcements/article/45695