Testing Guidelines

The purpose of this document is to help you get your application and App Store entry ready for QA testing.

Application Release Workflow (ARW)

Each application submitted to the Onshape App Store goes through a series of stage-gates:

1. Starting state: Ok to deploy to limited visibility on Production (Beta testing)
2. Ok to make Public
3. Goal state: Application is Public

To advance to the next stage, your application must pass testing, and your App Store entry must pass review.

Kick Off Testing

While completing the Launch Checklist, you will need to use Jira to request testing and release for your application. You can initiate the following tasks from Jira:

• Request application testing: This puts your application in the testing queue. We will note when testing has started (in progress) and when concluded, the ticket will be closed. The outcome will include notes and links to any issues generated (tickets). This phase may include as many iterations as needed to get your application ready.
• Request public (general) authorization: We will note when testing has started (in progress), and this request will trigger a review of your app store entry and any outstanding bugs. Note there is no implied testing of your application, simply a review of outstanding issues (tickets) and of the App Store entry. Success at this stage will advance the Application Release Workflow (ARW), and you can request public release.
• Request public (general) release: This request states that you have coordinated with the Developer Relations team and the Onshape Marketing team, and agreement has been reached that the app is ready for launch. Congratulations!

Testing Protocol

Applications are first tested against the checklist in Addendum A. Production App Store entries are then performed against the checklist in Addendum B.

Results will be viewable in your Onshape support system (i.e., Zendesk, Jira). The result of each test will be one of:

• Pass:
  • No action needed.
  • No notification issued.
• Enhancement: Suggestions we believe would make the application better.
  • Will NOT prevent the application from being turned on for public access.
• Bug (low priority): Slight deviations from the criteria that have low end user impact.
  • Will NOT prevent the application being turned on for public access.
  • No stipulated time-frame for resolution.
• Bug (medium priority): Material deviations from the criteria that are noticeable to the end-user. Represents a minor problem that requires a work-around.
  • Will NOT prevent the application from being turned on for public access.
  • Must be fixed within 30 days.
• Bug (high priority): Significant deviation from the criteria.
  • WILL prevent the application from being turned on for public access.
• Bug (MUST FIX): Significant deviation from protocol or security violation.
  • WILL prevent the application from being turned on for public access.
  • If the application is already public, it may be temporarily suspended from the App Store.

Testing Notes

• Testing may be requested at any time.
• Testing is done on a first-come basis.
• When testing is complete (pass or fail), you go to the back of the queue.

Addendum A

Application Test Criteria

• The application must use the Onshape OAUTH mechanism
• The OAUTH must be against the correct stack
• To be promoted to the Production stack, and hosted service must be on a monitored production server with worldwide 24/7 availability.
• The application should not generate any avoidable console (browser) errors
• The application should should provide one or more of the following options. The user should not have to leave the registration workflow to complete a pre-requisite.
  • Sign in using the Onshape ID (account created silently on first use)
  • Sign in with partner product account credentials
  • Create a new partner account
• The application must be capable of managing/displaying documents in excess of 20. The application must display reasonable performance when reading documents, workspaces, elements, and parts. At scale, an account may have thousands of documents, many with multiple workspaces and each with multiple elements. Suggested strategies include:
  • Using a Next button to load the next 20 documents
  • Using infinite scroll (loading the next 20 if the scrollbar reaches the bottom of the dialog)
  • Displaying the most recently-opened documents first
  • Displaying a counter of documents/workspaces/elements read
  • Using progressive loading
• The application should correctly list valid documents when per document app access is turned on.
• The application should correctly handle selection of versions.
• The application should correctly handle selection of workspaces (branches).
• The application should correctly handle/display elements that are:
  • Part Studios that contain nothing
  • Assemblies that contain nothing
  • Part Studios that contain only surfaces
  • Part Studios that contain only wire data (e.g., helices)
• The application should appropriately handle revocation of a grant.

Addendum B

App Store Testing Criteria

• The application should have a descriptive name
• The application summary should be accurate
• The redirect URLS should be valid
• The iframe URL should be valid
• The Grant (permissions) request should be no more than is needed
• The Application Type should be correctly set
• Team visibility should be set (optional)
• The category should be appropriate
• The application description should be accurate
• The Sign-In URL should be valid
• The pricing summary should be accurate
  • i.e., trials should not be listed as Free; Free for xx days and then $xx/month is more accurate.
• All pay plans should have accurate descriptions.
• The support URL should point to a resource for help (the resource should NOT be an FAQ page, unless that page also contains one of the other options):
  • Support ticketing system (e.g., Zendesk, Jira, etc.)
  • Web page with a telephone number
  • Web page with an email address
  • Forum
• The EULA link should point to an English Language EULA.