Integrating IDS NXT cameras into factory automation via OPC UA

To demonstrate controlling an IDS NXT camera via OPC UA, this tech tip describes an example AI image processing workflow that you can easily reproduce yourself with your IDS NXT camera and a few objects ( ballpoint pen, ruler, eraser, key, etc.). To do this, simply follow the instructions below step by step.

1) Setup inference task

• For this example we use the ” Classifier ” vision app with the preinstalled neural network (CNN) “ImageNet1000″. Install and start the app via the ” Vapp Manager ” in the IDS NXT cockpit.
• The classifier recognises many objects, such as ballpoint pens, keys, business cards, through the pre-trained ImageNet1000. Place any objects under your IDS NXT camera and setup image capture in freerun mode so that your objects are fully imaged with appropriate settings (colour, sharpness, exposure, etc.).
• Use ROIs for the objects you want to identify. The inference results of the classification of each image capture are displayed at the ROIs and in the results area with the respective probability.

In our case, a ballpoint pen (68%), a sharpener (100%) and an eraser (98%) are recognised.

2) Start OPC UA server

• You do not need to program anything to provide OPC communication on the part of the camera. Simply install (1) and start (2) the OPC UA server plugin via the “VAPP Manager” in the IDS NXT cockpit on your IDS NXT camera.
• You can start (1) and stop (2) the OPC UA server via the actions of the plugin. By default, the server is started automatically (3) when the plugin is started.

3) Create OPC UA configuration and recipe

• Creating and saving configurations on the IDS NXT camera via the OPC UA plugin is easy via the IDS NXT cockpit.
• Simply name the current camera settings (configuration name) and the inference task with the classifier (recipe name) and save the configuration and recipe via the corresponding actions. With this, you have already completed the OPC UA setup of the camera via the cockpit.

4) Establish OPC UA client – server connection

• Install and start the OPC UA client “UaExpert®”.
• To connect the OPC UA server of the camera, click on the (Add Server) in the menu bar. Switch to “Advanced” tab.
• Give the connection a “Configuration Name” Specify the “Endpoint URL” of the server (e.g. “opc.tcp://169.254.173.231:4840”)
  – Protocol: opc.tcp
  – IP address of the camera
  – OPC UA server port: 4840
• Select the desired “Security Policy” (supported are: None, Basic128Rsa15, Basic256, Basic256Sha256)
• Select “Message Security Mode” (None, Sign, Sign&Encrypt)
• Enter the access data for your IDS NXT camera in the “Authentication Settings”. (e.g. admin or service user)
• Now you can connect with the OPC UA server: “Connect”

5) Request / edit device information

• To test the server connection, you can navigate through the information model of the camera in the “Address Space” window. For example, if you change the device name in UaExpert®, it will also be updated in the IDS NXT cockpit.
• Furthermore, you can subscribe to status and statistics parameters via the ” Data Access View” to track their changes. If, for example, you drag the image counter “ImageTotal” into the window, its value will increase dynamically with each new image captured by the camera.
• To reset the image counter, execute the “Call” in the context menu of the method ” ResetAll”. The subscribed frame counter then starts counting again at “0”.

6) Activate a camera configuration

• First retrieve a list of the configurations stored on the camera with a call to the function GetConfigurationList.
• ResultCount shows us “1” available configuration with the InternalId “conf_classifier.json”.
• Use the function ActivateConfiguration to define the currently valid configuration whose saved parameters are then used in the camera. Set the corresponding InternalId as the input value (Id) for the call.
• You can check the currently active configuration via the node Active Configuration. The Id ” conf_classifier.json” should appear there in the attributes.

7) Prepare task (recipe)

We now proceed in the same way with the pre-saved image processing task of the camera.

• A call of the function GetRecipeListFiltered with the input value IsPrepared = 2 (DONTCARE_2) returns “1” stored recipe as ResultCount. In our case the saved recipe with the Id “recipe_classifier_imagenet.json”.
• With the function PrepareRecipe you now prepare this recipe as an active image processing task by setting the previously queried Id as the input value “InternalIdIn”.

8) Subscribe to a result event

The camera is now fully prepared via the OPC UA client. Before you execute the camera task, the result event must first be subscribed to.

• To do this, add an EventView…
• and drag ‘n drop the camera’s VisionSystem into this view.
• Activate the SimpleEvents and ConditionTypes fields to get all the details.
• Confirm the selection via the Apply button.

9) Execute task (recipe) – Evaluate event result

Now we let the camera do its job exactly once.

• To do this, call the method StartSingleJob once.
• A ResultReadyEvent is received from the camera in the EventView .
• In the event details, we can see the inference results of the three specified ROIs (ballpoint, pencil_sharpener, rubber_eraser) with the respective probabilities of classification.