Class Portkey

Example

const model = new Portkey({
  mode: "single",
  llms: [
    {
      provider: "openai",
      virtual_key: "open-ai-key-1234",
      model: "text-davinci-003",
      max_tokens: 2000,
    },
  ],
});

// Stream the output of the model and process it
const res = await model.stream(
  "Question: Write a story about a king\nAnswer:"
);
for await (const i of res) {
  process.stdout.write(i);
}

Hierarchy

BaseLLM
- Portkey

Index

Constructors

constructor

new Portkey(init?): Portkey
Parameters
- Optional init: Partial<Portkey>
Returns Portkey
Overrides BaseLLM.constructor
- Defined in docs/api_refs/langchain/src/llms/portkey.ts:100

Properties

CallOptions

CallOptions: BaseLLMCallOptions

ParsedCallOptions

ParsedCallOptions: Omit<BaseLLMCallOptions, never>

caller

caller: AsyncCaller

The async caller should be used by subclasses to make any async calls, which will thus benefit from the concurrency and retry logic.

session

session: PortkeySession

verbose

verbose: boolean

Whether to print out response text.

`Optional` apiKey

apiKey?: string = undefined

`Optional` baseURL

baseURL?: string = undefined

`Optional` cache

cache?: BaseCache<Generation[]>

`Optional` callbacks

callbacks?: Callbacks

`Optional` llms

llms?: null | [LLMOptions] = undefined

`Optional` metadata

metadata?: Record<string, unknown>

`Optional` mode

mode?: string = undefined

`Optional` tags

tags?: string[]

Accessors

callKeys

get callKeys(): string[]
Keys that the language model accepts as call options.

Returns string[]
Inherited from BaseLLM.callKeys
- Defined in docs/api_refs/langchain/src/base_language/index.ts:143

Methods

batch

batch(inputs, options?, batchOptions?): Promise<string[]>
Default implementation of batch, which calls invoke N times. Subclasses should override this method if they can batch more efficiently.
Parameters
- inputs: BaseLanguageModelInput[]
  
  Array of inputs to each batch call.
- Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]
  
  Either a single call options object to apply to each batch call or an array for each call.
- Optional batchOptions: RunnableBatchOptions & {
  returnExceptions?: false;
  }
Returns Promise<string[]>
An array of RunOutputs, or mixed RunOutputs and errors if batchOptions.returnExceptions is set
Inherited from BaseLLM.batch
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:159
batch(inputs, options?, batchOptions?): Promise<(string | Error)[]>
Parameters
- inputs: BaseLanguageModelInput[]
- Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]
- Optional batchOptions: RunnableBatchOptions & {
  returnExceptions: true;
  }
Returns Promise<(string | Error)[]>
Inherited from BaseLLM.batch
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:165
batch(inputs, options?, batchOptions?): Promise<(string | Error)[]>
Parameters
- inputs: BaseLanguageModelInput[]
- Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]
- Optional batchOptions: RunnableBatchOptions
Returns Promise<(string | Error)[]>
Inherited from BaseLLM.batch
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:171

bind

bind(kwargs): Runnable<BaseLanguageModelInput, string, BaseLLMCallOptions>
Bind arguments to a Runnable, returning a new Runnable.
Parameters
- kwargs: Partial<BaseLLMCallOptions>
Returns Runnable<BaseLanguageModelInput, string, BaseLLMCallOptions>
A new RunnableBinding that, when invoked, will apply the bound args.
Inherited from BaseLLM.bind
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:67

call

call(prompt, options?, callbacks?): Promise<string>
Convenience wrapper for generate that takes in a single string prompt and returns a single string output.
Parameters
- prompt: string
- Optional options: string[] | BaseLLMCallOptions
- Optional callbacks: Callbacks
Returns Promise<string>
Inherited from BaseLLM.call
- Defined in docs/api_refs/langchain/src/llms/base.ts:359

generate

generate(prompts, options?, callbacks?): Promise<LLMResult>
Run the LLM on the given prompts and input, handling caching.
Parameters
- prompts: string[]
- Optional options: string[] | BaseLLMCallOptions
- Optional callbacks: Callbacks
Returns Promise<LLMResult>
Inherited from BaseLLM.generate
- Defined in docs/api_refs/langchain/src/llms/base.ts:298

generatePrompt

generatePrompt(promptValues, options?, callbacks?): Promise<LLMResult>
This method takes prompt values, options, and callbacks, and generates a result based on the prompts.
Parameters
- promptValues: BasePromptValue[]
  
  Prompt values for the LLM.
- Optional options: string[] | BaseLLMCallOptions
  
  Options for the LLM call.
- Optional callbacks: Callbacks
  
  Callbacks for the LLM call.
Returns Promise<LLMResult>
An LLMResult based on the prompts.
Inherited from BaseLLM.generatePrompt
- Defined in docs/api_refs/langchain/src/llms/base.ts:182

getNumTokens

getNumTokens(content): Promise<number>
Parameters
- content: MessageContent
Returns Promise<number>
Inherited from BaseLLM.getNumTokens
- Defined in docs/api_refs/langchain/src/base_language/index.ts:201

invocationParams

invocationParams(_options?): any
Get the parameters used to invoke the model
Parameters
- Optional _options: Omit<BaseLLMCallOptions, never>
Returns any
Inherited from BaseLLM.invocationParams
- Defined in docs/api_refs/langchain/src/llms/base.ts:207

invoke

invoke(input, options?): Promise<string>
This method takes an input and options, and returns a string. It converts the input to a prompt value and generates a result based on the prompt.
Parameters
- input: BaseLanguageModelInput
  
  Input for the LLM.
- Optional options: BaseLLMCallOptions
  
  Options for the LLM call.
Returns Promise<string>
A string result based on the prompt.
Inherited from BaseLLM.invoke
- Defined in docs/api_refs/langchain/src/llms/base.ts:66

map

map(): Runnable<BaseLanguageModelInput[], string[], BaseLLMCallOptions>
Return a new Runnable that maps a list of inputs to a list of outputs, by calling invoke() with each input.

Returns Runnable<BaseLanguageModelInput[], string[], BaseLLMCallOptions>
Inherited from BaseLLM.map
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:78

pipe

pipe<NewRunOutput>(coerceable): RunnableSequence<BaseLanguageModelInput, Exclude<NewRunOutput, Error>>
Create a new runnable sequence that runs each individual runnable in series, piping the output of one runnable into another runnable or runnable-like.
Type Parameters
- NewRunOutput
Parameters
- coerceable: RunnableLike<string, NewRunOutput>
  
  A runnable, function, or object whose values are functions or runnables.
Returns RunnableSequence<BaseLanguageModelInput, Exclude<NewRunOutput, Error>>
A new runnable sequence.
Inherited from BaseLLM.pipe
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:461

predict

predict(text, options?, callbacks?): Promise<string>
This method is similar to call, but it's used for making predictions based on the input text.
Parameters
- text: string
  
  Input text for the prediction.
- Optional options: string[] | BaseLLMCallOptions
  
  Options for the LLM call.
- Optional callbacks: Callbacks
  
  Callbacks for the LLM call.
Returns Promise<string>
A prediction based on the input text.
Inherited from BaseLLM.predict
- Defined in docs/api_refs/langchain/src/llms/base.ts:376

predictMessages

predictMessages(messages, options?, callbacks?): Promise<BaseMessage>
This method takes a list of messages, options, and callbacks, and returns a predicted message.
Parameters
- messages: BaseMessage[]
  
  A list of messages for the prediction.
- Optional options: string[] | BaseLLMCallOptions
  
  Options for the LLM call.
- Optional callbacks: Callbacks
  
  Callbacks for the LLM call.
Returns Promise<BaseMessage>
A predicted message based on the list of messages.
Inherited from BaseLLM.predictMessages
- Defined in docs/api_refs/langchain/src/llms/base.ts:392

serialize

serialize(): SerializedLLM
Returns SerializedLLM

Deprecated
Return a json-like object representing this LLM.
Inherited from BaseLLM.serialize
- Defined in docs/api_refs/langchain/src/llms/base.ts:421

stream

stream(input, options?): Promise<IterableReadableStream<string>>
Stream output in chunks.
Parameters
- input: BaseLanguageModelInput
- Optional options: Partial<BaseLLMCallOptions>
Returns Promise<IterableReadableStream<string>>
A readable stream that is also an iterable.
Inherited from BaseLLM.stream
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:225

streamLog

streamLog(input, options?, streamOptions?): AsyncGenerator<RunLogPatch, any, unknown>
Stream all output from a runnable, as reported to the callback system. This includes all inner runs of LLMs, Retrievers, Tools, etc. Output is streamed as Log objects, which include a list of jsonpatch ops that describe how the state of the run has changed in each step, and the final state of the run. The jsonpatch ops can be applied in order to construct state.
Parameters
- input: BaseLanguageModelInput
- Optional options: Partial<BaseLLMCallOptions>
- Optional streamOptions: Omit<LogStreamCallbackHandlerInput, "autoClose">
Returns AsyncGenerator<RunLogPatch, any, unknown>
Inherited from BaseLLM.streamLog
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:507

toJSON

toJSON(): Serialized
Returns Serialized
Inherited from BaseLLM.toJSON
- Defined in docs/api_refs/langchain/src/load/serializable.ts:148

toJSONNotImplemented

toJSONNotImplemented(): SerializedNotImplemented
Returns SerializedNotImplemented
Inherited from BaseLLM.toJSONNotImplemented
- Defined in docs/api_refs/langchain/src/load/serializable.ts:222

transform

transform(generator, options): AsyncGenerator<string, any, unknown>
Default implementation of transform, which buffers input and then calls stream. Subclasses should override this method if they can start producing output while input is still being generated.
Parameters
- generator: AsyncGenerator<BaseLanguageModelInput, any, unknown>
- options: Partial<BaseLLMCallOptions>
Returns AsyncGenerator<string, any, unknown>
Inherited from BaseLLM.transform
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:478

withConfig

withConfig(config): RunnableBinding<BaseLanguageModelInput, string, BaseLLMCallOptions>
Bind config to a Runnable, returning a new Runnable.
Parameters
- config: BaseCallbackConfig
  
  New configuration parameters to attach to the new runnable.
Returns RunnableBinding<BaseLanguageModelInput, string, BaseLLMCallOptions>
A new RunnableBinding with a config matching what's passed.
Inherited from BaseLLM.withConfig
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:107

withFallbacks

withFallbacks(fields): RunnableWithFallbacks<BaseLanguageModelInput, string>
Create a new runnable from the current one that will try invoking other passed fallback runnables if the initial invocation fails.
Parameters
- fields: {
  fallbacks: Runnable<BaseLanguageModelInput, string, BaseCallbackConfig>[];
  }
  - fallbacks: Runnable<BaseLanguageModelInput, string, BaseCallbackConfig>[]
    
    Other runnables to call if the runnable errors.
Returns RunnableWithFallbacks<BaseLanguageModelInput, string>
A new RunnableWithFallbacks.
Inherited from BaseLLM.withFallbacks
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:124

withRetry

withRetry(fields?): RunnableRetry<BaseLanguageModelInput, string, BaseLLMCallOptions>
Add retry logic to an existing runnable.
Parameters
- Optional fields: {
  onFailedAttempt?: RunnableRetryFailedAttemptHandler;
  stopAfterAttempt?: number;
  }
  - Optional onFailedAttempt?: RunnableRetryFailedAttemptHandler
  - Optional stopAfterAttempt?: number
Returns RunnableRetry<BaseLanguageModelInput, string, BaseLLMCallOptions>
A new RunnableRetry that, when invoked, will retry according to the parameters.
Inherited from BaseLLM.withRetry
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:88

`Static` deserialize

deserialize(data): Promise<BaseLLM<BaseLLMCallOptions>>
Parameters
- data: SerializedLLM
Returns Promise<BaseLLM<BaseLLMCallOptions>>

Deprecated
Load an LLM from a json-like object describing it.
Inherited from BaseLLM.deserialize
- Defined in docs/api_refs/langchain/src/llms/base.ts:438

`Static` isRunnable

isRunnable(thing): thing is Runnable<any, any, BaseCallbackConfig>
Parameters
- thing: any
Returns thing is Runnable<any, any, BaseCallbackConfig>
Inherited from BaseLLM.isRunnable
- Defined in docs/api_refs/langchain/src/schema/runnable/base.ts:557

Class Portkey

Example

Hierarchy

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Optional init: Partial<Portkey>

Returns Portkey

Properties

CallOptions

ParsedCallOptions

caller

session

verbose

Optional apiKey

Optional baseURL

Optional cache

Optional callbacks

Optional llms

Optional metadata

Optional mode

Optional tags

Accessors

callKeys

Returns string[]

Methods

batch

Parameters

inputs: BaseLanguageModelInput[]

Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

Optional batchOptions: RunnableBatchOptions & { returnExceptions?: false; }

Returns Promise<string[]>

Parameters

inputs: BaseLanguageModelInput[]

Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

Optional batchOptions: RunnableBatchOptions & { returnExceptions: true; }

Returns Promise<(string | Error)[]>

Parameters

inputs: BaseLanguageModelInput[]

Optional options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

Optional batchOptions: RunnableBatchOptions

Returns Promise<(string | Error)[]>

bind

Parameters

kwargs: Partial<BaseLLMCallOptions>

Returns Runnable<BaseLanguageModelInput, string, BaseLLMCallOptions>

call

Parameters

prompt: string

Optional options: string[] | BaseLLMCallOptions

Optional callbacks: Callbacks

Returns Promise<string>

generate

Parameters

prompts: string[]

Optional options: string[] | BaseLLMCallOptions

Optional callbacks: Callbacks

Returns Promise<LLMResult>

generatePrompt

Parameters

promptValues: BasePromptValue[]

Optional options: string[] | BaseLLMCallOptions

Optional callbacks: Callbacks

Returns Promise<LLMResult>

getNumTokens

Parameters

content: MessageContent

Returns Promise<number>

invocationParams

Parameters

Optional _options: Omit<BaseLLMCallOptions, never>

Returns any

invoke

Parameters

input: BaseLanguageModelInput

`Optional` init: Partial<Portkey>

`Optional` apiKey

`Optional` baseURL

`Optional` cache

`Optional` callbacks

`Optional` llms

`Optional` metadata

`Optional` mode

`Optional` tags

`Optional` options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

`Optional` batchOptions: RunnableBatchOptions & {
returnExceptions?: false;
}

`Optional` options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

`Optional` batchOptions: RunnableBatchOptions & {
returnExceptions: true;
}

`Optional` options: Partial<BaseLLMCallOptions> | Partial<BaseLLMCallOptions>[]

`Optional` batchOptions: RunnableBatchOptions

`Optional` options: string[] | BaseLLMCallOptions

`Optional` callbacks: Callbacks

`Optional` options: string[] | BaseLLMCallOptions

`Optional` callbacks: Callbacks

`Optional` options: string[] | BaseLLMCallOptions

`Optional` callbacks: Callbacks

`Optional` _options: Omit<BaseLLMCallOptions, never>

`Optional` options: BaseLLMCallOptions

`Optional` options: string[] | BaseLLMCallOptions

`Optional` callbacks: Callbacks

`Optional` options: string[] | BaseLLMCallOptions

`Optional` callbacks: Callbacks

`Optional` options: Partial<BaseLLMCallOptions>

`Optional` options: Partial<BaseLLMCallOptions>

`Optional` streamOptions: Omit<LogStreamCallbackHandlerInput, "autoClose">

fields: {
fallbacks: Runnable<BaseLanguageModelInput, string, BaseCallbackConfig>[];
}

`Optional` fields: {
onFailedAttempt?: RunnableRetryFailedAttemptHandler;
stopAfterAttempt?: number;
}

`Optional` onFailedAttempt?: RunnableRetryFailedAttemptHandler

`Optional` stopAfterAttempt?: number

`Static` deserialize

`Static` isRunnable