Structured Storage

Sub Elements

Onshape provides application elements storage that is controlled by applications through the API. These elements allow a set of named sub-elements.

The application can make changes to sub-elements independently or in arbitrary groupings. Changes may be wholesale replacements, or may be deltas. When performing a delta update, the application may post a full version as well, which allows the api to return a smaller number of deltas for subsequent queries.

An application may need to perform multiple versionable actions in the course of performing a user-level action, and we want to allow the individual actions to be collected into a single action from the perspective of document history. We do this by providing support for creation of a private transaction and support for atomically committing the transaction to the document workspace as a single user-visible action.

Onshape does not assume any knowledge about the semantics of application deltas. All merging of deltas into a consolidated form is done by the application. Applications should typically send checkpoint state for a sub-element if many delta changes have been made since the last checkpoint.

Document content and changes are logically an array of bytes, but since they are transmitted through JSON, then are expected to be presented a Base-64 encoding of the array into string form.

We use some terminology in this document that is new.

  • changeId - an opaque identifier for the state of an application element. Each change to the application element results in a new changeId

  • transaction - a private workspace within a document workspace for composing modifications to an application element. These changes are not visible to the user until committed.

  • transaction commit - an operation that moves the changes performed within a transaction to the application element workspace as a single user-visible action.

Concurrent access by multiple users

If the element is being concurrently accessed by multiple sessions, updates may encounter conflicts during update. If the application has a mechanism that ensures that all accesses to the element are mediated by a single process, as is done with our part studio and assemblies, this can be addressed directly by the application. However, if the application is not able to mediate access in this way, updates by one session may invalidate state held by another session. We address this by notifying updaters when an update cannot be directly applied because their state is out of date and allowing them to refresh their state before re-applying the change.

This policy of requiring the application have current state when posting updates could be overly conservative in some cases. Detecting conflict at the sub-element level might provide for better concurrent access performance, but there probably are cases where this fails, so it probably would need some level of application control.

JSON Tree

In contrast with sub elements, JSON tree storage is a more managed data storage mechanism that Onshape itself can merge and diff. At the root of it, the data structure is a single JSON object per Application Element. The user submits incremental changes that are then applied by Onshape to the JSON tree. Onshape stores these ‘diffs’ in a new microversion created as a result of the update request, or during a subsequent transaction commit request. When the user then performs a merge or restore operation, Onshape can sum and apply the requisite incremental changes. By storing diffs, Onshape provides to the user a storage mechanism that is more robust to race conditions, since multiple simultaneous edits can be optionally merged by Onshape. All of these qualities make JSON tree a preferred way to store application element data in an Onshape-native manner.

JSON Tree Edit Semantics

BTJEdit Encoding

A JSON tree edit represents an incremental change to an application element’s JSON tree. The edit is a BTJEdit class, which is encoded as one of the following:

  • Deletion:
{ "btType" : "BTJEditDelete-1992", "path" : "path" }
  • Insertion:
{ "btType" : "BTJEditInsert-2523", "path" : "path", "value" : "newValue" }
  • Change:
{ "btType" : "BTJEditChange-2636", "path" : "path", "value" : "newValue" }
  • Move:
{ "btType" : "BTJEditMove-3245", "sourcePath" : "path", "destinationPath" : "path" }
  • List (where edit1, edit2, etc. are zero or more edits.):
{ "btType" : "BTJEditList-2707", "edits" : [ "edit1", "edit2", "..."] }

Within the above encoding, newValue is a stand in for any valid JSON, and path is a stand in for an object representing a path to the node at which to perform the edit.

BTJPath Encoding

The BTJPath object describes a path through the JSON tree to a particular node, and is encoded as follows:

{ "btType" : "BTJPath-3073", "startNode" : "startNode", "path" : [ "pathElement1", "pathElement2", "..."] }

where startNode is a string that is either empty to specify the root node or a nodeId of a node in the tree. The pathElement is one of:

  • Key:
{ "btType" : "BTJPathKey-3221", "key" : "string" }
  • Index:
{ "btType" : "BTJPathIndex-1871", "index" : "integer" }

In the insertion and move type edits the path elements can describe a path that doesn’t currently exist. Onshape will generate the proper keys and values as needed to place the node value in the proper location.

JSON Tree Examples

Below are some examples that show the body required to perform the particular edit on a JSON tree.

Deletion Example

If the pre-existing JSON tree looks like:

{"myKey":  "myValue"}

and a delete edit looks like:

{"btType": "BTJEditDelete-1992",
     "path": {"btType": "BTJPath-3073", "startNode": "", "path": [{"btType": "BTJPathKey-3221", "key": "myKey"}]}}

then the resulting JSON is the result of deleting the node specified by path:

{}

Insert Example

If the pre-existing JSON tree looks like:

{}

and the insertion edit looks like:

{"btType": "BTJEditInsert-2523",
     "path": {"btType": "BTJPath-3073", "startNode": "", "path": [{"btType": "BTJPathKey-3221", "key": "insertedKey"}]},
     "value": "myValue"}

then the resulting JSON is the result of inserting the node described by value at the node specified by path:

{"insertedKey": "myValue"}

Change Example

If the pre-existing JSON tree looks like:

{"myKey":  "myValue"}

and the change edit looks like:

{"btType": "BTJEditChange-2636",
     "path": {"btType": "BTJPath-3073", "startNode": "", "path": [{"btType": "BTJPathKey-3221", "key": "myKey"}]},
     "value": "myOtherValue"}

then the resulting JSON is the result of changing the node specified by path to the node described by value:

{"myKey":  "myOtherValue"}

Move Example

If the pre-existing JSON tree looks like:

{"myKey":  "myValue", "myOtherKey":  "myOtherValue"}

and the move edit looks like:

{"btType": "BTJEditMove-3245", "sourcePath": {"btType": "BTJPath-3073", "startNode": "",
                                                  "path": [{"btType": "BTJPathKey-3221", "key": "myKey"}]},
     "destinationPath": {"btType": "BTJPath-3073", "startNode": "",
                         "path": [{"btType": "BTJPathKey-3221", "key": "keyCreatedFromMove"}]}}

then the resulting JSON is the result of moving the node from the specified sourcePath to the destinationPath:

{"keyCreatedFromMove": "myValue"}

List Example

If the pre-existing JSON tree looks like:

{}

and the list edit looks like:

{"btType": "BTJEditList-2707", "edits": [
        {"btType": "BTJEditInsert-2523", "path": {"btType": "BTJPath-3073", "startNode": "", "path": [{"btType": "BTJPathKey-3221", "key": "myKey"}]},
         "value": "myValue"},
        {"btType": "BTJEditChange-2636", "path": {"btType": "BTJPath-3073", "startNode": "",
                                                  "path": [{"btType": "BTJPathKey-3221", "key": "myKey"}]},
         "value": ["firstValue", "secondValue"]},
        {"btType": "BTJEditInsert-2523", "path": {"btType": "BTJPath-3073", "startNode": "",
                                                  "path": [{"btType": "BTJPathKey-3221", "key": "myKey"},
                                                           {"btType": "BTJPathIndex-1871", "index": 1}]},
         "value": "myBetterSecondValue"},
        {"btType": "BTJEditDelete-1992", "path": {"btType": "BTJPath-3073", "startNode": "",
                                                  "path": [{"btType": "BTJPathKey-3221", "key": "myKey"},
                                                           {"btType": "BTJPathIndex-1871", "index": 2}]}}
    ]}

then the resulting JSON is the result of applying all the given edits in order:

{"myKey": ["firstValue", "myBetterSecondValue"]}

The intermediate steps were:

  1. Insertion:
       {"myKey":  "myValue"}
  2. Change:
       {"myKey": ["firstValue", "secondValue"]}
  3. List insertion:
       {"myKey": ["firstValue", "myBetterSecondValue", "secondValue"]}
  4. List deletion:
       {"myKey": ["firstValue", "myBetterSecondValue"]}

All the examples above were tested and validated using the Python client here.