{"id":20676,"date":"2019-10-04T10:23:00","date_gmt":"2019-10-04T10:23:00","guid":{"rendered":"https:\/\/www.inogic.com\/blog\/?p=20676"},"modified":"2019-12-10T13:16:03","modified_gmt":"2019-12-10T13:16:03","slug":"power-platform-how-to-build-ai-object-detector-using-power-platform","status":"publish","type":"post","link":"https:\/\/www.inogic.com\/blog\/2019\/10\/power-platform-how-to-build-ai-object-detector-using-power-platform\/","title":{"rendered":"Power Platform: How to Build AI Object Detector using Power Platform"},"content":{"rendered":"

Introduction<\/strong><\/h2>\n

In this AI Builder three-part blog series, first blog<\/a> was about how to use AI Builder to extract Form document data and second blog<\/a> highlighted the use of AI Builder Text classification model.<\/p>\n

Now in the third and final blog we will see how to build an AI Object Detector using Power Platform.<\/p>\n

Object detection\u00a0refers to the capability of computer and software systems to detect images, locate\u00a0objects\u00a0in an image and identify each\u00a0object.\u00a0Object detection can be used to expedite or automate business processes in multiple industries.<\/p>\n

AI Object detection is a new feature in Power Platform which can be accomplished using AI Builder. AI Builder allows you to add artificial intelligence to your apps even if you have no coding or data science skills.<\/p>\n

Let\u2019s see how an AI object detection model is built using AI builder. On the other hand, Object detection App is built using PowerApps and data is transferred from PowerApps to Dynamics 365 CRM using Microsoft Flow in Power Platform.<\/p>\n

1. Build Object Detection Model using AI Builder<\/strong><\/p>\n

a. To build a model using AI Builder, sign in to PowerApps and, in the left navigation pane, select AI Builder > Build<\/strong>. Then select the\u00a0Object Detection<\/strong>\u00a0model type.<\/p>\n

\"How<\/p>\n

b. Name your model and Create<\/strong><\/p>\n

c. Select object names<\/strong> – Select the object names by selecting an entity from Common Data Service. Here, we have selected a custom entity with name \u2018Electronic Item\u2019<\/strong> and Field \u2018Name\u2019<\/strong>. This entity has records with name of electronic items like Monitor, Keyboard, Mobile and Mouse.<\/p>\n

\"How<\/p>\n

\"How<\/p>\n

d. Add images<\/strong> – Click on \u2018+Add Images\u2019<\/strong> button to upload images of objects that you want to detect and identify. Upload 15+ images of each object.<\/p>\n

\"How<\/p>\n

e. Tag images<\/strong>– Teach the model to identify objects by tagging and labeling them. For each image, draw a rectangle around the object which you want to identify and tag it with correct name. Go through all the images to tag the objects. Once all the images have been tagged, click on Done tagging<\/strong>.<\/p>\n

\"How<\/p>\n

f. Train Model <\/strong>\u2013 Click on Next >Train<\/strong>. It will take several minutes to train the model. Use Quick Test<\/strong> to test the model\u2019s accuracy and then Publish<\/strong> the model to consume it in the app.<\/p>\n

Once the model is published its status changes to Live<\/strong> from Draft.<\/p>\n

\"How<\/p>\n

\"How<\/p>\n

2. Build Object Detector App using PowerApps<\/strong><\/p>\n

a. Click on Create app<\/strong> button shown in previous step. You can create a Canvas App or Model-driven App or Portal from blank depending upon your choice. In our example, we are creating a Mobile Canvas App.<\/p>\n

b. Insert Object Detector by navigating to Insert > AI Builder > Object detector. <\/strong>Drag the Object detector to your screen and select the existing object detector AI model.<\/p>\n

\"How<\/p>\n

The object detector control looks like below:<\/p>\n

\"How<\/p>\n

c. Insert a Gallery Control with Title and subtitle layout.<\/p>\n

\"How<\/p>\n

Set the properties of Gallery and its Title and SubTitle as follows:-<\/p>\n

Gallery Items\u00a0 = ObjectDetector1.VisionObjects<\/p>\n

Title Text = ThisItem.displayName<\/p>\n

Subtitle Text = ThisItem.count<\/p>\n

d. Insert some label controls for storing the values of Count for each individual item i.e. Monitor, Keyboard, Mobile and Mouse Counts and set their text properties.<\/p>\n

\"How<\/p>\n

For Example: –<\/p>\n

lblKeyboard.Text =First(ObjectDetector1.VisionObjects).count<\/p>\n

lblMobile.Text= First(Filter(ObjectDetector1.VisionObjects,StartsWith(displayName,”Mobile”))).count<\/p>\n

lblMonitor.Text= First(Filter(ObjectDetector1.VisionObjects,StartsWith(displayName,”Monitor”))).count<\/p>\n

lblMouse.Text= Last(ObjectDetector1.VisionObjects).count<\/p>\n

You can make your own formula by utilizing the Gallery or Object Detector controls<\/p>\n

e. Insert a Create button. Click on Create button, navigate to Action > Flows > Select the existing flow or Create a new Flow <\/strong>as described in 3rd<\/sup> point. <\/strong>It will start processing and add the selected flow to PowerApp.<\/p>\n

\"How<\/p>\n

Modify the function of Onselect as shown below by passing the parameters to the flow. Here, CreateElectronicEntry is the name of flow.<\/p>\n

OnSelect =CreateElectronicEntry.Run(Concat(Gallery1.AllItems,displayName & “-” & count & “,”),lblMobile.Text,lblKeyboard.Text,lblMonitor.Text,lblMouse.Text)<\/p>\n

f. Save and Publish the App.<\/strong><\/p>\n

3. Pass data from PowerApps to Dynamics 365 CRM using Microsoft Flow<\/strong><\/p>\n

You can pass data to Dynamics 365 CRM using the Patch statement on click button in PowerApps itself but here we are showing it using a flow because Microsoft flow has many capabilities and you can perform so many tasks by connecting your app to different connectors depending on your business requirements.<\/p>\n

a. Create a new instant Flow with trigger of PowerApps.<\/p>\n

b. Add New Step >> Search for an Action \u2018Common Data Service\u2019 >> Create a New record.<\/p>\n

c. Select your \u2018Environment\u2019 and \u2018Entity Name\u2019. We have created another custom entity \u2018Electronic Entry\u2019 which has the fields Monitor, Keyboard, Mobile and Mouse which shows the count of these devices.<\/p>\n

d. Map the fields by clicking on \u2018Ask in PowerApps\u2019. It will automatically map relevant PowerApps fields with the CRM fields.<\/p>\n

e. Save the flow.<\/p>\n

\"How<\/p>\n

4. Run the App <\/strong><\/p>\n

You can capture an image from your mobile camera or select an existing image from Gallery. Click on the Detect button. The objects will be identified if it matches with trained objects and the list would show count of each object detected in the image.<\/p>\n

\"AI\"<\/p>\n

When Create button is clicked it triggers the flow which creates a record in CRM as shown below: –<\/p>\n

\"How<\/p>\n

Conclusion<\/strong><\/h2>\n

In this way, you can build AI Object detection App which\u00a0is an easy to use tool, just point and click, to tailor\u00a0AI\u00a0in your business\u00a0as per your requirements. Object Detection\u00a0lets you to count, locate and identify selected\u00a0objects\u00a0within any image. You can\u00a0use\u00a0this model in\u00a0PowerApps\u00a0to extract information from pictures you take\u00a0with\u00a0the camera.<\/p>\n

\"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Introduction In this AI Builder three-part blog series, first blog was about how to use AI Builder to extract Form document data and second blog highlighted the use of AI Builder Text classification model. Now in the third and final blog we will see how to build an AI Object Detector using Power Platform. Object\u2026 Read More »<\/a><\/span><\/p>\n","protected":false},"author":13,"featured_media":20692,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1901,18,19,1913],"tags":[1956,1931],"class_list":["post-20676","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-builder","category-dynamics-365-v9-2","category-dynamics-crm","category-microsoft-power-platform-services","tag-ai-builder","tag-microsoft-power-platform"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/posts\/20676","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/comments?post=20676"}],"version-history":[{"count":0,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/posts\/20676\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/media\/20692"}],"wp:attachment":[{"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/media?parent=20676"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/categories?post=20676"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inogic.com\/blog\/wp-json\/wp\/v2\/tags?post=20676"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}