(Go: >> BACK << -|- >> HOME <<)

Page MenuHomePhabricator

Function model change: add keys to type, add serializers and deserializers
Closed, ResolvedPublic

Description

We will need the following further keys on Z4 for the next few features we want to provide:

  • Z4K4/equality - type Z8 - a function that takes two instances of the type at hand and returns a Boolean, whether they are the same.
  • Z4K5/renderer - type Z8 - a function that takes an instance of the type at hand and a language and returns a string given a short textual representation of the given instance
  • Z4K6/parser - type Z8 - a function that takes a string and a language, and turns the given string into an instance if possible
  • Z4K7/deserializers - Z810(Z46), list of deserializers - deserializers takes a ZObject and returns a value in the given programming language (see below)
  • Z4K8/serializers - Z810(Z64), list of serializers - serializers take a value of a specific type in that programming language and return a ZObject (see below)

We also add two new types, for Deserializers and Serializers:

Z46/Deserializer

  • Z46K1/id: points to itself (i.e. type Z46)
  • Z46K2/type: of type Z4 (type to be deserialized)
  • Z46K3/programming language: of type Z61
  • Z46K4/target type: of type Z6 (the name of the type in the given programming language)
  • Z46K5/code: of type Z6, the actual code that takes the ZObject in some serialized form and returns a value of type Z46K4

Z64/Serializer

  • Z64K1/id: points to itself (i.e. type Z64)
  • Z64K2/type: of type Z4 (type to be serialized into)
  • Z64K3/programming language: of type Z61
  • Z64K4/source type: of type Z6 (the name of the type in the given programming language)
  • Z64K5/code: of type Z6, the actual code that takes the value of type Z64K4 and returns a ZObject for internal translation

Details

TitleReferenceAuthorSource BranchDest Branch
Update function-schemata sub-module to HEAD (57b1cf7)repos/abstract-wiki/wikifunctions/function-evaluator!129jforrestersync-function-schematamain
Update function-schemata sub-module to HEAD (57b1cf7)repos/abstract-wiki/wikifunctions/wikilambda-cli!24jforrestersync-function-schematamain
Update function-schemata sub-module to HEAD (57b1cf7)repos/abstract-wiki/wikifunctions/function-evaluator!127jforrestersync-function-schematamain
Update function-schemata sub-module to HEAD (57b1cf7)repos/abstract-wiki/wikifunctions/function-orchestrator!110jforrestersync-function-schematamain
definitions: Add K4/equality, K5/renderer, and K6/parser keys to Z4/Typerepos/abstract-wiki/wikifunctions/function-schemata!62jforresterT346991main
Update function-schemata sub-module to HEAD (b5a37af)repos/abstract-wiki/wikifunctions/wikilambda-cli!16jforrestersync-function-schematamain
Update function-schemata sub-module to HEAD (b5a37af)repos/abstract-wiki/wikifunctions/function-orchestrator!72apinesync-function-schematamain
Update function-schemata sub-module to HEAD (b5a37af)repos/abstract-wiki/wikifunctions/function-evaluator!62apinesync-function-schematamain
data: Define Z46/Deserialiser and Z64/Serialiser typesrepos/abstract-wiki/wikifunctions/function-schemata!43jforresterT346991main
Add Z46 and Z64, Deserializers and Serializers.repos/abstract-wiki/wikifunctions/function-schemata!42apineapine-add-de-derializersmain
Customize query in GitLab

Event Timeline

I have some quibbles about Z46K4 and Z64K4. These are concerns for much later down the line, so I'm really blustering over nothing, but I think it's important that we be aware of this issue.

Having the type name as a string can be very useful when serializing. Currently, we use the introspection capabilities of JS and Python to decide how to serialize: we look at type(object) or typeof Object, then perform a string comparison (or something similar). If we see that the object is the custom type ZPair, for example, we then serialize as an instance of Z882. So these Strings will work well for now.

However, the name of the type, given as a string, will not generally suffice for type inference in all programming languages. In the worst case, we may need to provide yet another function (in the implementation language) to identify whether an in-memory object is of a given type.

We could, of course, pass the return type to the evaluator and just blindly serialize the output as that type. We used to do this. There are some complications (e.g., for container types), but we've already decided that (de)serialization functions will, for now, not recursively (de)serialize the members of a ZObject. Therefore, passing the return type will work unless the Z8's return type is Z1. In that case, we still need to identify the type of the return value in order to serialize.

In summary: for cases where

  1. a function return a Z1 and
  2. a programming language lacks the introspection capabilities to get a string representation of an object's type,

we need a general solution for type inference.

That said, we don't support any such languages now and won't for a while, so starting with string representations of the type's name will work for a long time :).

That is a great point.

My first thought would be that we forbid native code implementations for functions that return Z1, but that sounds a bit draconian.

But it would allow us to skip reflection and just rely on the declared output type. (Modulo errors and exceptions?)

On the other side, how many useful functions with a Z1 return type are there really out there? I can't think of many, most of which are probably builtins.

Hmm. Maybe it is true that programming languages which a weaker type system tend to have reflection. In a stronger typed language though, we wouldn't need reflection.

Just thinking. One way or the other, this doesn't look like a blocker.

I agree that it's not a blocker at all, just something I wanted on the record.

I am also wondering what the function's name will be. We'll need to inject that as a string into the executor so that it knows what function to call.

For Z8/Functions with native code implementations, we currently accomplish this by tying the function name to the Z8K5/Identity. So, if a Z8/Function has the ZID Z1000, the code string has to be something like def Z1000(...)... or function Z1000 (...) {...}.

I guess here we could use Z46K1 and Z64K1? Can we make it a constraint that the function defined be named identically to the ZID of the Z46 or Z64 object?

Yes, Z46K1 and Z64K1 is meant to be the identity, i.e. have the same constraints as Z8K5

Hmm, one more thought. Since we already have the Z16/Code object, should we use that instead of having both Z64K3 and Z64K5 (likewise Z46K3 and Z46K5)?

Change 966271 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (b5a37af)

https://gerrit.wikimedia.org/r/966271

Jdforrester-WMF changed the task status from Open to In Progress.Oct 17 2023, 2:20 PM
Jdforrester-WMF moved this task from Backlog to In Progress on the Abstract Wikipedia team board.

Change 966271 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (b5a37af)

https://gerrit.wikimedia.org/r/966271

Change 980889 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] [WIP] Add initial support for Z4K4–K8, Z46, and Z64 for type support

https://gerrit.wikimedia.org/r/980889

Change 981590 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (57b1cf7)

https://gerrit.wikimedia.org/r/981590

Change 981590 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (57b1cf7)

https://gerrit.wikimedia.org/r/981590

Change 980889 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Add initial support for Z4K4–K8, Z46, and Z64 for type support

https://gerrit.wikimedia.org/r/980889

Change 992225 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2023-11-29-152839 to 2024-01-18-182456

https://gerrit.wikimedia.org/r/992225

Change 992227 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2023-11-29-143341 to 2024-01-18-182630

https://gerrit.wikimedia.org/r/992227

Change 992225 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2023-11-29-152839 to 2024-01-18-182456

https://gerrit.wikimedia.org/r/992225

Change 992229 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2023-11-29-152839 to 2024-01-09-190638

https://gerrit.wikimedia.org/r/992229

Change 992229 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2023-11-29-152839 to 2024-01-09-190638

https://gerrit.wikimedia.org/r/992229

Change 992227 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2023-11-29-143341 to 2024-01-18-182630

https://gerrit.wikimedia.org/r/992227