Skip to main content
View in Github The ProgramState class is the core object of Enochian. The core functions are .add, .user, .assistant, .system, and .gen. The ProgamState has an internal list of Message that defines the state of the object. The idea is that the ProgramState should only be affected by .add, and the other role functions and generate functions can use the state of the ProgramState but should never change any of it’s messages.

​
.add

Parameters:
  • message: Message or
  • message: Promise<Message> or
  • message: AsyncGenerator<Message, Message, undefined>
Returns:
  • ProgramState or
  • Promise<ProgramState> or
  • AsyncGenerator<Message>
Types Referenced: Adds a Message object to the internal state of the ProgramState. It can either be a Message, a Promise<Message>, or an AsyncGenerator<Message, Message, undefined>. If it’s just a Message then the .add function will be synchronous and return a this, so you can chain together calls. If it’s a Promise<Message>, it will await on the Message, add it to it’s internal state, then return a Promise<this>. This is usually when you add a role template with a .gen inside of it, but it can be any arbitrary function that returns a Promise<Message>. Lastly, if it’s an AsyncGenerator<Message, Message, undefined> then it will yield results until the generator is done, then add it to the internal state.

​
.user, .assitant, .system

Parameters:
  • strings: TemplateStringsArray, ...values: Stringifiable[] or
  • strings: TemplateStringsArray, ...values: ((messages: Message[]) => Promise<string> | Promise<Stringifiable> | Stringifiable)[] or
  • strings: TemplateStringsArray, ...values: ((messages: Message[]) => AsyncGenerator<string, void, unknown> | (messages: Message[]) => Promise<string> | Promise<Stringifiable> | Stringifiable)[]
Returns:
  • Message or
  • Promise<Message> or
  • AsyncGenerator<Message>
Types Referenced:
  • Message
  • type Stringifiable = string | number | boolean | bigint;
Creates a Message object with a role corresponding to the function name. The template function can take in interpolated values of Stringifiable, (messages: Message[]) => Promise<string>, and (messages: Message[]) => AsyncGenerator<string, void, unknown>. If the interpolated values only consist of Stringifiable, then the role function will be synchronous and return a Message. The Stringifiable objects will all be appended to the prompt with their .toString() methods. For these synchronous functions, the function can be passed to .add and the .add function will be synchronous as well. If the interpolated values have any (messages: Message[]) => Promise<string>, then it will call that function and await it’s result. For now, it expects that function to always be the .gen function. If the interpolated values have any (messages: Message[]) => AsyncGenerator<string, void, unknown>, then it will return an AsyncGenerator<Message> and yield the chunks as they come in. It expects the function to always be the .gen function that was called with stream: true. The goal of the role functions is to return a Message without affecting the state of the ProgramState. If you want to add the Message, it can only be done with .add and that is intentional.

​
.gen

Parameters
  • answerKey: string, genInput?: Omit<GenerateReqNonStreamingInput, 'text' | 'input_ids'> or
  • answerKey: string, genInput?: Omit<GenerateReqStreamingInput, 'text' | 'input_ids'>
Returns
  • (messages: Message[]) => Promise<string> or
  • (messages: Message[]) => AsyncGenerator<string, void, unknown>
Types Referenced: The .gen function is basically like OpenAI’s .chat.completions api. The only difference is that calling it returns a function that calls the generate endpoint. The reason it’s done like this is that when you interpolate the .gen call inside a role function, you want the messages that .gen is passing to the generate endpoint to be up to date with the currently processing string template. For example, if the string template looks something like s.assistant`Sure! Here's your ${s.gen(...)}`;, then the interpolated .gen should know that it should prepend "Sure! Here's your " to the assistant message. If .gen simply return a Promise<string>, then it would start processing with the state of the ProgramState at the moment it was called, and not when it should actually start processing.

​
.fromSGL

Parameters
  • url: string
Returns
  • Promise<ProgramState>
Updates the ProgramState backend to use an SGLang server running at the provided url.

​
.fromOpenAI

Parameters
  • opts?: { client?: ClientOptions; modelName?: OpenAI.ChatModel; baseURL?: string; }
Returns
  • ProgramState
Updates the ProgramState backend to use OpenAI with the provided options.

​
.fork

Parameters
  • _numForks?: number
Returns
  • ProgramState[]
Deep copies the program state and returns _numForks copies. If _numForks is not passed in or is less than 0, it forks once.

​
.get

Parameters
  • key: string or
  • key: string, { from: "zod": schema: z.ZodType } or
  • key: string, { from: "tools": tools: ToolUseParams }
Returns
  • string | undefined or
  • z.infer<ZodType> | undefined or
  • ToolUseResponse | undefined
Gets the value that was generated for a given answerKey that was passed into .gen. If the key is found and no additional options were passed in, then it will return a string. If it is called with a "zod" option, then it will use the passed in schema to parse it and return the resulting type. This should only really be used when you passed in a schema to .gen in the SamplingParams to do constrained decoding. If it is called with a "tools" option, then it will return in the shape of the in tools definition. Returns undefined if the key does not exist.

​
.getPrompt

Parameters None Returns
  • string
Applies the chat template to the current messages in the ProgramState and returns it.

​
.getMetaInfo

Parameters
  • key: string
Returns
  • MetaInfo | undefined
Types Referenced: Gets the meta info for a given answerKey that was passed into .gen. Returns undefined if the key does not exist.

​
.setDebugInfo

Parameters
  • debugInfo: Partial<Debug> | null
Returns
  • ProgramState
Types Referenced: Set’s the debug info for enochian-studio to log. In most cases you should prefer .beginDebugRegion and .endDebugRegion though, this one should be reserved for changing the url or port that enochian-studio is served from.

​
.beginDebugRegion

Parameters
  • debugInfo: { debugName: string; debugPromptID?: string }
Returns
  • ProgramState
This marks the beginning of a debug region in your code. Any calls to the generate endpoint will be logged and grouped under the debugName. Most of the time you will never have to set the debugPromptID as it’s automatically generated if it’s not passed in, but you can if you want to.

​
.endDebugRegion

Parameters None Returns
  • ProgramState
This marks the end of a debug region.

​
Tools

You are able to pass in a ToolUseParams to .gen to give the LLM the ability to call tools.

​
ToolUseParams

export function createTools<
    const T extends Array<{
        function: (...args: any[]) => any;
        name: string;
        params?: z.ZodSchema;
        description?: string;
    }>,
>(
    tools: T,
): { [K in keyof T]: Tool<T[K]['name'], ReturnType<T[K]['function']>> } {
    return tools as any;
}
export type ToolUseParams = ReturnType<typeof createTools>;
This is a complicated way to represent this type: 
type ToolUseParams = {
    function: Function;
    name: string;
    params?: z.ZodSchema;
    description?: string;
}[]
You must use createTools to create a ToolUseParams if you want .get to be able to infer the function names and return types of those functions.

​
ToolUseResponse

type ToolUseResponse = ({
    toolUsed: string;
    response: unknown;
} | {
    toolUsed: "respondToUser";
    response: string;
})[]
This type represents the result of a function call. The LLM could either call one or more tools OR respond to the user directly. The reason you should use the createTools util is so that ToolUseResponse can infer the literal value for toolUsed as the name of the function that was called, and response as the return type of the function that was called.