Dynamic schemas for Apple's on-device LLM

In the article about tools for Apple's on-device LLM we've seen how to implement an MCP-like tool for Apple's on-device LLM. The schema for the tool's input was defined at compile time as follows:

@Generable
struct Arguments {
    var latitude: Double
    var longitude: Double
}

Sometimes it is useful to dynamically define schemas at run time. For example, one might want to generate the schema from an OpenAPI specification. The above schema can be defined for the tool as follows:

let parameters: GenerationSchema
    
init() throws {
    let schema = DynamicGenerationSchema(name: "arguments", properties: [
        DynamicGenerationSchema.Property(name: "latitude", schema: DynamicGenerationSchema(type: Double.self)),
        DynamicGenerationSchema.Property(name: "longitude", schema: DynamicGenerationSchema(type: Double.self)),
    ])
    parameters = try GenerationSchema(root: schema, dependencies: [])
}

We just need to provide the properties' names and types. Here we use Double for both the latitude and longitude.

The signature of the tool's method remains the same but we have to indicate what the arguments' type is:

typealias Arguments = GeneratedContent

func call(arguments: Arguments) async throws -> Reply { ...

The GeneratedContent structure is a dynamic container for the data generated by the LLM before it invokes our tool.

The properties can be accessed as follows:

let latitude = try arguments.value(Double.self, forProperty: "latitude")
let longitude = try arguments.value(Double.self, forProperty: "longitude")

The value method simply returns the corresponding property and converts it into the explicitly provided data type.

In this way it is possible to use tools created at run time with Apple's on-device LLM.