Welcome to Code Forum!

Join a community that supports you and your coding journey from day one. We strive to be a friendly, supportive community that empowers everyone to be better developers. By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!
  • Guest, before posting your code please take these rules into consideration:
    • It is required to use our BBCode feature to display your code. While within the editor click < / > or >_ and place your code within the BB Code prompt. This helps others with finding a solution by making it easier to read and easier to copy.
    • You can also use markdown to share your code. When using markdown your code will be automatically converted to BBCode. For help with markdown check out the markdown guide.
    • Don't share a wall of code. All we want is the problem area, the code related to your issue.

    GIF shows where to locate </> in the thread and or post editor toolbar.
    To learn more about how to use our BBCode feature, review our "How to post your code into threads" here.

    Thank you, Code Forum.

AI Local Models

simong1993

simong1993

Diamond Coder
Staff Team
Guardian
Hey All,

So I've been experimenting with Local AI, I'll queue Antero's response :laugh: . So at the moment I use 14 local AIs, Lama, Deepseek etc etc all free, I then use Claude to find the best AI/Prompt now i think ive got this down but would anyone be intrested in making our own local AI, something we can build to connect to our day to day life, think Iron man.
 
Well, I would like to but I don't know how AI's work or how to build one. Good luck finding someone advanced enough to build one. X E.
 
Play Snayk 3+, higher dimensional snake game?
Being my first colab project, this would be a learning curve for me too 😀 So im thinking first building the base program, something with a UI, you can ask questions. it can work out the best local AI to answer. then maybe building extensions so say a question was about turning on a light switch its coded to understand

or do SEO its got a base to run by, maybe on github open source so others can make tools that can be intergrated if that makes sense
 
Just a note, be prepared to use big storage to do this other people, like at least a few GBs, he is using 400GBs on 16 models. X E.
 
Play Snayk 3+, higher dimensional snake game?
Im bit confused here.

​

simong1993, do you mean:
a. Training a LLM from a start?
b. Fine-tuning some current LLM for certain purpose?
c. Creating a GUI/TUI to work with some LLM?

If
a. Possible but bit hard and tricky. Needs powefull pc and few gigabytes of training data, atleast even for minor model.
b, it is actually pretty easy if you just have training data. Even average laptop with 8gigs can do it. My desktop does not even sweat for a such a task.
c. There is already tons of them, a good ones.
End if
 
simong1993 said:
Hey All,

So I've been experimenting with Local AI, I'll queue Antero's response :laugh: . So at the moment I use 14 local AIs, Lama, Deepseek etc etc all free, I then use Claude to find the best AI/Prompt now i think ive got this down but would anyone be intrested in making our own local AI, something we can build to connect to our day to day life, think Iron man.
Click to expand...
Hell yeah brother, I like jarvis, but well should also have like an ai like cortanna from halo.

With that being said Local ai is kind of a doosie, when you say local AI, do you mean an AI that we can download and use on our own, or use on the website? That will kinda define how hard it is to do.

If we are making a downloadable LLM or AI we are going to need a launcher(You need and Xbox to play halo) I would suggest using kobold because she is open source and the love of my life. She(I choose what gender an AI launcher is) can only launch GGUF files which helps us out figuring what type of format to make our ai/LLM in. Working on a chat bot would be the """"""Easiest"""""" thing to do first. We would start by ripping a already Working model, Take out that neural network(Organizational binder), and train it on data sets of our choosing.

The Next degree of operations would be to devolop our own data sets, which involves a crawler(a collector of data across the web) and a scrubber(a formater that makes data readable for our AI, and also removes posionios data) and then we train our data with our own parameters. IE do we want a Libarian that can quote things, or do we want novelist paraphraser. these data sets can range from 1-100 Terabytes. Gaurd rails come in there somewhere but I forget where.(They make the bot who read one hockey book-tok novel not go into heat when someone asks "what does a goalie look like?")

After that it is all hypothetical(I have no Idea)
 
I went AWOL, sorry guys, I didn't see the messages 😀 So my idea was to use local AIs, so prebuilt ones. At the moment, I have a product bot, so I use 1 for categories, 1 for titles, 1 to make the prompts for the other AIs, one to make sure the output is right etc etc so it got me thinking Deepseek is amazing at Code, Llama is good at chat, so why couldnt we merge them all togther. have say dolphine work out what the question was then pass it to the other correct AI then have another AI turn the question and answer into a smaller version so as it grows it learns more about you 😀
 
Play Snayk 3+, higher dimensional snake game?
The world does not yet work quite the same way as in the movies 😉

Retraining an already trained local model is not possible.

It is possible to perform so-called fine-tuning, which does not actually give the model new information, but rather controls how the model responds.
So the model does not actually learn/remember anything about you.

Currently, local models are usually 1b - 13b. In them, the size somewhat limits what they are good at.

Some in coding, some in general conversation.

I recommend getting acquainted with methods such as RAG and agents, there are opportunities there that can help you get closer to what I understand you are talking about.
 
simong1993 said:
Hey All,

So I've been experimenting with Local AI, I'll queue Antero's response :laugh: . So at the moment I use 14 local AIs, Lama, Deepseek etc etc all free, I then use Claude to find the best AI/Prompt now i think ive got this down but would anyone be intrested in making our own local AI, something we can build to connect to our day to day life, think Iron man.
Click to expand...
BUT WHY
 

New Threads

Latest posts

Share this page

Buy us a coffee!

Buy me a coffee.
Code Academy Advertisement
DataCamp Advertisement

About Us

Code Forum is a community platform where coding enthusiasts can connect with other developers, engage in discussions, ask for help, and share their knowledge with a supportive community. It's a perfect place to improve your coding skills and to find a community of like-minded individuals who share your passion for coding.

Site links

Stay Connected

Back
Top Bottom