C# GetOutput("logits") API

[ambroser53]

Is there any reason that the C# API wouldn't have the same interface for retrieving the logits that the python API (#956) and C API (#755) have? With the exposing of the adapters API #998 this would be incredibly helpful.

Also #956 introduced the set logits interface although I haven't seen much discussion of it's use cases. I may be wrong but if this allows for the force setting of tokens mid-generation then this would be incredibly useful (for example, generate tokens, check condition, add n tokens, generate more tokens). The current C# API does not currently (unless I am mistaken) allow for the altering of input_ids within the gen_params mid-inference. Is there an alternate method to doing this? Or is the set logits interface the appropriate method?

[ambroser53]

Realised the second half of my query is being addressed in "continuous decoding" as referenced in #828 and #967 so the request would have been a c# interface which I see is currently underway in aciddelgado/continuous.

Regardless a method of obtaining logits within the C# API would be useful.

[baijumeswani]

I think #998 adds that API.
Note that the get logits API (in #956) is different than the get outputs ("logits") API in that one of them get's you the logits as used by the search and sampling functions and the other gets you the raw model outputs.

[ambroser53]

You're right I missed that, that's great. Do you know which is which? I'm assuming the GetOutput in #998 is to get the raw model outputs correct? And the getoutput("logits") is the search and sampling logits? What processing takes place for the search and sampling logits?