How Power Platform Global AI #HackTogether inspired AI-powered solutions for real-world use cases
Seeking inspiration for the year in building AI-powered solutions? We’ve summarized a collection of solutions by use cases from last September’s Power Platform Global AI #HackTogether. Over the course of 2 weeks,
- There were several live sessions (now available on-demand) for participants to learn about the Copilot and AI features of Power Platform.
- Participants followed a Learn Collection for help in getting started.
- Over 100 project submissions were received with 4 winners chosen across the different categories.
Summary of the participants and project submissions from around the world.
The categories for the hackathon were:
Grand prize winner – The best of the best! Rewards the solution that meets all judging criteria, wows judges, and has potential real-world value for Power Platform Users.
Best AI-powered solution – Rewards the solution that integrates AI in an innovative way.
Best Better Together Use Case – Rewards the solution that uses the Power Platform and other Microsoft products in an interesting way.
Best Diversity, Equity, and Inclusion (DEI) Use Case – Rewards the solution that helps make the Power Platform community more diverse, equitable and inclusive.
The submissions were innovative and demonstrated how AI can be helpful across numerous use cases. Be inspired this year in building AI-powered solutions with Power Platform by checking out these community submissions below:
- Education – Teaching Accelerator
- Family Support – Make Life Easy | Parents Advisor
- Healthcare – Romão’s Homecare | Medical Assistant
- Human Resources – The Relocation Game | Seeker
- Not-for-profit – Migrant Worker Case Management
- Information Technology – Prompt Wagon | CodeScribe
Education
Teaching Accelerator
Teachers spend thousands of hours each year planning the curriculum for their students. Elliot Fraser developed a Power Platform solution, known as the Teaching Accelerator, to expedite the process for teachers in creating a curriculum for their students. The solution utilizes OpenAI’s ChatGPT service where teachers can interact with it through a Copilot embedded in the model-driven app to create lessons, modify the lessons created, and create an entire curriculum of lessons for a topic.
Teachers can interact with a Copilot within the model-driven app for creating lessons and a curriculum for students.
The solution consists of the following main features:
- Model-driven app which is the primary end user interface for the teachers to plan their lessons and a curriculum for students.
- Custom pages built using Power Apps canvas apps are embedded in the model-driven app to help guide teachers to create a curriculum of lessons, create lesson plans and view, modify and assign existing lessons.
- Teachers enter their questions into a Copilot that was built using Microsoft Copilot Studio to receive AI-generated suggestions for a curriculum and lesson plans. The Copilot appears in a pane within the model-driven app through a custom page.
- Power Automate cloud flows in the background interact with OpenAI’s ChatGPT service to relay the suggestions to the teacher.
- When the teacher is satisfied with the suggestions from Copilot, they can ask Copilot to generate the curriculum and lesson plans in the model-driven app. Depending on where the teacher executes this in the app, Copilot calls a function in OpenAI to create an array of the curriculum or lesson plans as a JSON object. This output is then saved in Dataverse using a cloud flow by transforming it into rows of data which the teacher sees as the curriculum or lesson plans in the model-driven app.
What we loved
Elliot’s Teaching Accelerator solution won the Grand Prize winner category of the hackathon due to the impact it would have in the education space by reducing the hours teachers spend annually. Teachers can quickly create a curriculum and lesson plans for students in minutes with the help of generative AI and Power Platform.
Since the hack, Elliot has an updated version of the Teaching Accelerator where he’s made enhancements based on feedback from teachers who have been testing the solution. For an overview of the updated solution version, watch Elliot’s presentation from a previous The Low Code Revolution episode.
Family support
Make Life Easy
To provide everyday support for parents of autistic children, Raghav Mishra developed a Power Apps canvas app to create structured daily routines. As a parent to an autistic child, Raghav recognized their son thrived when steps of their daily routines were illustrated visually by his wife with the corresponding written text. Inspired by his wife’s dedication in helping their son, Raghav turned to the Power Platform combined with OpenAI’s services to develop a Power Apps canvas app that uses generative AI to help create the text-based steps of the tasks and design the corresponding images of the steps. The Make Life Easy app can be used by parents to establish routines the children can follow.
Parents interact with generative AI in a canvas app to create written and visual steps for tasks children follow.
The solution consists of the following main features:
- Parents use the canvas app to interact with OpenAI’s ChatGPT service to generate a list of text-based steps for tasks, such as the steps children follow to put their shoes on.
- The text-based steps for each task are saved in Dataverse, which are displayed in the gallery of the app.
- Parents can select a step from the gallery and interact with OpenAI’s DALL-E service to create the corresponding visual images for all steps, such as asking for an image of a child sitting on a chair as a step the child follows to tie their shoelaces.
- Once all images of the steps for the tasks are created, they can be added to the schedule board for children to follow. Parents can then assign the tasks to their children.
- Children see their assigned tasks in their personal daily schedule board within the app. They can follow each of the steps through the visual and written instructions generated by AI.
What we loved
Raghav’s Make Life Easy solution won the Best Diversity, Equity and Inclusion (DEI) Use Case category for designing a solution that supports parents of autistic children and their daily routines. Parents can interact with generative AI in a Power Apps canvas app to define the steps of a task and create the corresponding visuals. Children can follow these AI-generated steps through written and visual comprehension in the app.
Parents Advisor
To help parents encourage and teach children every day, Richard Li created a Power Apps canvas app known as Parents Advisor, where parents can seek additional support for advice on educating their children. Several Azure OpenAI services and OpenAI services were used together with Power Platform to provide tailored recommendations based on the individual in-app user profiles. Children can also have one-on-one interaction with AI, where the AI-generated answers are verbally read to them from the app itself. The Parents Advisor app can be used daily by both parents and children for written and aural comprehension.
Parents and children interact with generative AI to create tailored learning experiences in the canvas app.
The solution consists of the following main features:
- Parents can select from built-in prompts in the canvas app such as how to encourage their child to sleep independently. They can also enter their own custom prompts directly in the app, where Azure OpenAI’s ChatGPT service will respond with an answer which the parent can save for future reference.
- Parents also have the option to provide their prompt verbally through the built-in microphone control of canvas apps. Their verbal prompt is then transcribed to text using OpenAI’s Whisper speech-to-text service and can also be translated to more than 50 languages.
- All responses from Azure OpenAI’s ChatGPT service can be transcribed from text-to-speech using the Speech service from Azure AI. The style of the voice can be altered using Speech Synthesis Markup Language (SSML) such as the effect, style and multilingual voices. This means the AI advisor can “talk” to children by reading the responses to prompts through the built-in audio control of canvas apps.
- Most of the parent end-user features are also replicated for the children end-users where they too can interact with the Parent Advisor app and access the list of built-in topic prompts. Or they can enter their own custom prompts and have the app read the responses generated.
- One of the fun features of the app is using Azure OpenAI’s ChatGPT service to author bedtime stories for children based on pre-selected criteria of the character (e.g. Cinderella), language, genre, length and the requested story details. The parent or child also has an option to create their own bedtime story by adding their own prompts. Based on the prompts selected, the app will generate a bedtime story which can be read by the parent or child, or the app can read it to them through the native audio control of canvas apps.
What we loved
It was very cool to see the solution utilize the Speech service from Azure AI to transcribe the responses from text-to-speech in relation to the prompt entered by the parent or child. We also liked it was diverse and inclusive in providing multi-language support to cater for different languages of families.
Healthcare
DISCLAIMER: Applications built using Microsoft Power Platform are not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used to replace or as a substitute for professional medical advice, diagnosis, treatment, or judgment.
Romão’s Homecare
To assist doctors and nurses with scheduled visits to patient’s homes, the Romão brothers – Douglas and Renato, built several apps by combining low code and pro-code techniques. A Power Apps canvas app, known as the Romão’s Homecare mobile app, is used by the doctors and nurses for visibility of daily scheduled in-home patient visits and to record details of their assessment from their devices. The information entered in the Romão Homecare app is saved in Dataverse and is accessible through a Power Apps model-driven app.
Doctors and nurses capture their assessment of patients during in-home visits with assistance from generative AI in a canvas app.
The solution consists of the following main features:
- The canvas app is the primary end user interface for doctors and nurses where they can see a list of their upcoming scheduled in-home visits. The built-in interactive map control displays the location of their patient visits.
- The native camera control of canvas apps enables them to capture photos during their assessment with the patient. These photos are uploaded into Azure Blob Storage and a sentiment score based on the images uploaded is generated using a Python web app via Azure App service.
- Audio recordings can also be captured using the native microphone control in canvas apps where the audio is transcribed to text using OpenAI Whisper’s speech-to-text service.
- A simple Yes/No checklist is used which is submitted to OpenAI to provide a score of the visit.
- On completion of the visit, the overall score calculated by OpenAI indicates the wellbeing of the patient with a high score being good and a low score requiring further attention.
- A model-driven app displays the stored patient data in Dataverse where information can be modified if needed.
What we loved
Doctors and nurses taking notes during or after a patient visit takes time and the solution reduces this administrative effort by capturing audio recordings and transcribing them to text using OpenAI Whisper’s speech-to-text service. There’s no duplicate time spent re-entering their notes by typing into the app. The data being accessible through Dataverse is also convenient for the information to be modified if additional details need to be edited.
Medical Assistant
To help the medical industry interpret computerized tomography (CT) scans and magnetic resonance imaging (MRI) scans, Yevhenii Dementiev, Valentin Gasenko and Surkho Salamov built an app, known as the Detect Cancer app, to support physicians with reviewal of patient scans. The aim of the solution is to help physicians to distinguish between possible tumors and other anomalies, improving the precision of diagnosis and identifying tumors at various stages.
AI technologies used in their solution identifies potential tumors and anomalies to help physicians with diagnosis.
The solution consists of the following main features:
- Physicians use a canvas app where images from the CT and MRI scans can be uploaded using the built-in attachments control of canvas apps.
- Using AI Builder Object Detection Model, the uploaded images are analyzed and highlight the detected possible cancer regions on the CT scan and MRI scan.
- A Power Automate cloud flow sends the detected information from the AI model to OpenAI’s Completions service to generate medical conclusions for the physicians to predict the recovery of the patient.
- The physician can also dive further into the regions detected for analysis and comparison from previous CT and MRI scans of the patient. Based on the historic data, OpenAI can provide a prediction of the patient’s recovery from cancer.
- Dataverse is used to store all information about the patient scans and conclusions of the detected cancer regions.
What we loved
This is an incredible solution that supports a healthcare use case by assisting detection of cancer regions and medical conclusions using AI Builder Object Detection Model combined with OpenAI services. It was also nice to see the team include feedback in their video submission from their target audience of physicians. Their solution has demonstrated how Power Platform and AI together can be of secondary help to healthcare professionals.
Human Resources
The Relocation Game
Employee relocation can be overwhelming for employees in two ways: understanding what their organization’s local policies are and adjusting to a new city – especially if the employee is unfamiliar with the local surroundings. Denisa Mihai built a Power Apps canvas app known as The Relocation Game, to help employees with their move by interacting with a Copilot to learn about the local policies and help identify the city’s landmarks through AI Builder.
Employees interact with a Copilot embedded in a canvas app to learn more about their work benefits, the required documentation for relocation and can be connected to the HR team.
The solution consists of the following main features:
- A Copilot is embedded in a Power Apps canvas app which uses the built-in boost conversation capabilities where natural language processing is applied to find, collate and parse relevant information from the sites and summarize the search results into plain language for the employee.
- Employees interact with the Copilot within the Relocation app where they can ask about the company benefits, documentation required for their move to a new country, and request to be connected to the HR team for additional questions.
- An AI Builder object detection custom model was developed and trained to recognize city landmarks for employees to become familiar with their city’s surroundings. The trained object detection custom model is embedded into the Relocation app.
- Using the built-in camera and image controls of canvas apps, the employee takes photos and uploads these into the object detection custom model for it to recognize and present the name of the landmark to the employee.
- Employees can also create their own collection of landmarks by taking photos and adding them to their list of landmarks in the Relocation app.
What we loved
Employees can feel supported for their relocation to another city by interacting with a Copilot with boost conversation capabilities enabled to help with HR related enquiries. We liked that there was an element of “discovery” built into the app for employees to take photos and verify a landmark of the city using the AI Builder object detection custom model.
Seeker
Recruiting new talent is often time-consuming during the phases of advertising, finding candidates and interviewing candidates. Isabelle Gaboc, Daniel Kerridge, and Jady Mulqueeney developed an end-to-end solution to reduce the administrative hours spent. A Power Pages site was built to handle the job listings and the initial interview process with candidates. Candidates can choose to have an automated interview within the Power Pages site where interviews are tailored to the candidate. Azure OpenAI’s ChatGPT capabilities in AI Builder generates open ended questions in alignment to the job description and information provided in candidate’s resumes, and OpenAI’s ChatGPT service is utilized during the interview for an ongoing conversational dialog with the candidate.
Open ended interview questions generated with the AI Builder connector in Power Automate are used in the AI-led interviews with candidates.
The solution consists of the following main features:
- An AI-generated Power Pages site was created using Copilot that enables candidates to review all available positions. They can apply by uploading their resumes and begin the automated interview process shortly after.
- Several cloud flows were built to extract text from the uploaded resume where the data is parsed and analyzed using the AI Builder text recognition prebuilt model to convert the resume to text.
- The automation continues by converting the job description to text using Azure OpenAI GPT capabilities in AI Builder to format the resume text from the previous step by removing unnecessary information and only keep the information aligned to the job description. As the last automation step, AI Builder’s OpenAI GPT capabilities are used again to generate open ended interview questions based on the refined resume text and job description.
- The candidate interviews are conducted within a webpage in the Power Pages site through a virtual call with a .NET bot integrated with Azure Communication Services. The interview questions generated by AI Builder from the cloud flow are retrieved by the .NET bot from the webpage using liquid, a markup language in Power Pages.
- Using Azure AI services text-to-speech, the .NET bot asks the candidate the interview questions generated by AI Builder.
- In return, the candidate can respond verbally to the .NET bot where the audio dialog is translated from speech-to-text using Azure AI services.
- The .NET bot will then send the translated text to OpenAI’s ChatGPT service with a prompt to respond accordingly for the interview. The output used for the response is translated once again to speech and relayed aurally back to the candidate. This enables a constant conversational AI dialog loop between the bot and the candidate.
What we loved
Their Seeker site solution won the Best AI-powered solution category of the hackathon due to the impact it would have for Human Resources in allowing candidates to use a Power Pages site to upload their resume for a job application and use OpenAI’s ChatGPT services to create a list of interview questions, followed by conducting the interview through a virtual voice call – all without any human intervention.
For a more detailed overview of the solution watch the Seeker team’s presentation from a previous Microsoft 365 & Power Platform weekly call.
Not-for-profit
Migrant Worker Case Management
Filipino citizens who work abroad sometimes face difficulties in understanding their rights in the country they work in. A case management solution was developed by Carmina Symaco, Marcos Antonio Abrematia and Jayson Espadero to support migrant workers in solving their issues by interacting with a Copilot embedded in a Power Pages site. All questions are created as a case in Dataverse where additional support personnel can review and respond to the person working abroad.
Migrant workers can paste an excerpt from their employment contract into the page in the Power Pages site to generate details that are more easily understood.
The solution consists of the following main features:
- A Copilot built using Microsoft Copilot Studio utilizes the built-in boost conversation capabilities where local websites of a country are associated to the Copilot to generate responses to questions asked by migrant workers.
- The Copilot is embedded in a Power Pages site where the user provides details of their role, the country they are working in and selects from a list of topics that resemble their issue to gain further assistance. Based on the information provided, Copilot returns the desired information extracted and collated from local websites of the country the Filipino citizen works in.
- Excerpts from their employee contract can also be parsed and analyzed using Power Automate cloud flows combined with OpenAI’s ChatGPT service to interpret the details and generate an answer that can be easily understood by them.
- All answers are captured as a case where support personnel can review them in a canvas app. An embedded Copilot in the app assists the support personnel to generate responses for the cases.
- AI Builder text recognition prebuilt model was used to classify the cases based on the description the migrant worker provided to the Copilot in Power Pages. In the app, there is a view for cases where the tag identified by AI Builder and the selected issue type by the migrant worker do not match. The administrator reviews the conversation history between the migrant worker and Copilot to correct the classification of the case.
What we loved
The solution was supportive of migrant workers by using a Copilot with boost conversation capabilities to answer queries and use OpenAI’s ChatGPT service to help them understand their employment contracts. Since it’s embedded in a Power Pages site, they’re able to access the Copilot 24/7 which is of benefit to workers across different time zones. The team also included some neat features for the support personnel and one that stood out was using AI Builder text recognition prebuilt model to verify the classification of the case was correct which will help with the long-term quality of the data.
Information Technology
Prompt Wagon
Artem Chernevskiy, Katerina Chernevskaya, and Nikita Chernevskiy recognized that prompt engineering is a growing new skill for organizations of all industries to utilize. Prompt engineering is natural language inputs or queries users provide to Large Language Models (LLMs) to produce the desired outputs or responses. To help organizations become skilled in prompt engineering, a Power Apps canvas app known as Prompt Wagon, was developed where users can select from a list of more than 100 pre-defined prompts and practice their knowledge of prompts by interacting with Azure OpenAI’s ChatGPT service within the app. The aim of the app is to help end users of all levels – beginners or advanced to learn how to construct their prompts and grow their prompt engineering skills.
Users can practice prompt engineering with the canvas app and gain more confidence in using generative AI.
The solution consists of the following main features:
- Users can select from 100+ pre-defined prompts across three categories of Business, Data and Education in the canvas app to help them exercise prompt engineering with the Azure OpenAI ChatGPT service. It also includes multi-language support in English, German and French.
- A custom connector was built to query the Azure OpenAI ChatGPT service from the canvas app. This enables users to interact with the services from within the app.
- Dataverse is used to store the text, images, prompts and translations of the German and French languages. All of these are surfaced onto the canvas app from Dataverse.
- Built-in app settings are available to modify the prompts and query parameters directly within the app. The updated settings are adhered to when end users next enter their prompts into the app.
What we loved
The app will help organizations be better equipped with prompt engineering through interacting with generative AI in the app. It was great to see the team build a custom connector to make the actions reusable across their solution that query the Azure OpenAI ChatGPT service from the canvas app. The use of Dataverse for the high volume of text, images, prompts and translations of several languages will support the solution to scale as the library of prompts grow over time with use.
CodeScribe
Developers often review code authored by others, with a common scenario being legacy code, where there can be little documentation or comments in the source code. To help automate code reviews, Robert Perillo built a Power Apps canvas app known as CodeScribe for developers to copy and paste code snippets into the app. A summary of the code is provided using Azure OpenAI’s Completions service to assist developers with understanding the code.
Developers can perform code reviews instantly by entering code snippets into a Power Apps canvas app and Azure OpenAI will summarize the code.
The solution consists of the following main features:
- Developers can paste code snippets into the canvas app and when they submit the code for review within the app, a Power Automate cloud flow executes to call the Azure OpenAI Completions service which analyzes the code snippet.
- Azure OpenAI then formats the code into two outputs, the first being a code summary that provides an overview of what the code does, and the second is a commented code block that explains each step of the code in detail for documentation purposes.
- The Power Automate cloud flow sends the formatted code to the canvas app for developers to review the summarized code and view a running commentary on the lines of code.
What we loved
Robert’s CodeScribe app won the Best Better Together Use Case category to help developers with the task of reviewing and documenting code by combining Power Platform and Azure OpenAI Completions service. We liked that the summaries are produced within minutes, reducing the time spent on code reviews. The solution breaks down what the code does by first summarizing it in a short paragraph, followed by a detailed explanation for each of the steps in the code.
Share your story with us!
Do you have a story or experience to share? We are excited to learn more about how individuals and organizations are using Power Platform with generative AI and Copilot capabilities! Submit your story at https://aka.ms/ShareAIStory
