{"id":1433,"date":"2018-06-04T22:49:12","date_gmt":"2018-06-05T05:49:12","guid":{"rendered":"https:\/\/officedevblogs.wpengine.com\/?p=1433"},"modified":"2021-08-27T15:35:26","modified_gmt":"2021-08-27T22:35:26","slug":"working-with-files-in-your-microsoft-teams-bot","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/working-with-files-in-your-microsoft-teams-bot\/","title":{"rendered":"Working with files in your Microsoft Teams bot (Preview)"},"content":{"rendered":"<p>We announced at Build 2018 that bots will soon be able to\u00a0<a href=\"https:\/\/techcommunity.microsoft.com\/t5\/Microsoft-Teams-Blog\/Build-and-manage-tailored-apps-for-the-enterprise-using-the\/ba-p\/190580\" target=\"_blank\" rel=\"noopener noreferrer\">send and receive files<\/a>. I\u2019m happy to let you know that it\u2019s finally here! If you have a Microsoft Teams app that works with files, it\u2019s time to\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/microsoftteams\/platform\/resources\/dev-preview\/developer-preview\" target=\"_blank\" rel=\"noopener noreferrer\">switch to Public Developer Preview<\/a>\u00a0and check it out.<\/p>\n<p>&nbsp;<\/p>\n<p>To help get you started, I\u2019ll explain how the feature works, and walk through a sample app that shows it in action. The bot uses the\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/Computer-vision\/Home#optical-character-recognition-ocr\" target=\"_blank\" rel=\"noopener noreferrer\">Azure Computer Vision API<\/a>\u00a0to recognize text in an image, then sends the text back to the user as a file. If you want to dive straight into code, the source for the sample is up on Github for both\u00a0<a href=\"https:\/\/github.com\/aosolis\/visionsample-msteams-node\" target=\"_blank\" rel=\"noopener noreferrer\">Node.js<\/a>\u00a0and\u00a0<a href=\"https:\/\/github.com\/aosolis\/visionsample-msteams-csharp\" target=\"_blank\" rel=\"noopener noreferrer\">C#<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<h2 id=\"toc-hId-1624535776\">Receiving files<\/h2>\n<p>Your bot can now receive file attachments in a 1:1 chat (previously, they were silently dropped from the message activity). The attachment JSON looks like this:<\/p>\n<pre>{\n    \"contentType\": \"application\/vnd.microsoft.teams.file.download.info\",\n    \"contentUrl\": \"https:\/\/&lt;onedrive_path&gt;\/phototest.tif\",\n    \"name\": \"phototest.tif\",\n    \"content\": {\n        \"downloadUrl\": \"https:\/\/&lt;onedrive_download_url&gt;\",\n        \"uniqueId\": \"70D29F4A-05B7-434A-B7E6-B651FBFEF508\",\n        \"fileType\": \"tif\"\n    }\n}<\/pre>\n<ul>\n<li><code>contentType<\/code>\u00a0is\u00a0<code>application\/vnd.microsoft.teams.file.download.info<\/code>.<\/li>\n<li><code>name<\/code>\u00a0is the name of the file.<\/li>\n<li><code>contentUrl<\/code>\u00a0is a direct link to the file on OneDrive for Business. Note that this is\u00a0<em>not<\/em>\u00a0how your bot will access the file.<\/li>\n<li><code>content.fileType<\/code>\u00a0is the file type, as deduced from the file name extension.<\/li>\n<li><code>content.downloadUrl<\/code>\u00a0is a pre-authenticated link to download the file.<\/li>\n<\/ul>\n<p>To fetch the contents of the file, send a GET request to the URL in\u00a0<code>content.downloadUrl<\/code>. The URL is only valid for a few minutes, so you must fetch the file immediately.<\/p>\n<h3 id=\"toc-hId--1124134690\">Inline images<\/h3>\n<p>Users have long been able to send an\u00a0<em>image<\/em>\u00a0to a bot by inserting it directly into the compose box. On desktop and web clients, the user copies the image content and pastes it into the compose box. On mobile, there is also a button to insert a picture from the photo library. Images sent this way are received as a different kind of attachment:<\/p>\n<pre>{\n    \"contentType\": \"image\/*\",\n    \"contentUrl\": \"https:\/\/smba.trafficmanager.net\/amer-client-ss.msg\/v3\/attachments\/&lt;id&gt;\/views\/original\"\n}<\/pre>\n<ul>\n<li><code>contentType<\/code>\u00a0starts with\u00a0<code>image\/<\/code>.<\/li>\n<li><code>contentUrl<\/code>\u00a0is a resource under the Bot Framework\u00a0<code>\/v3\/attachments<\/code>\u00a0API.<\/li>\n<\/ul>\n<h5 id=\"toc-hId-225648635\">OCR Bot<\/h5>\n<p>OCR Bot looks for an image in the incoming message, whether it&#8217;s a file attachment or inline. Notice that we get the image content differently in the two cases. For a file attachment, the bot can pass the pre-authenticated\u00a0<code>downloadUrl<\/code>\u00a0directly to the Computer Vision API. However, because the\u00a0<code>contentUrl<\/code>\u00a0of an inline image requires an access token, the bot downloads the image bytes first, and then sends them as input to the API.<\/p>\n<pre>\/\/ 1) File attachment: a file picked from OneDrive or uploaded from the computer\nconst fileAttachments = msteams.FileDownloadInfo.filter(session.message.attachments);\nif (fileAttachments &amp;&amp; (fileAttachments.length &gt; 0)) {\n    \/\/ Image was sent as a file attachment\n    \/\/ downloadUrl is an pre-authenticated URL to the file contents, valid for only a few minutes\n    const resultFilename = fileAttachments[0].name + \".txt\";\n    this.returnRecognizedTextAsync(session, () =&gt; {\n        return this.visionApi.runOcrAsync(fileAttachments[0].content.downloadUrl);\n    }, resultFilename);\n    return;\n}\n\n\/\/ 2) Inline image attachment: an image pasted into the compose box, or selected from the photo library on mobile\n\/\/ getFirstInlineImageAttachmentUrl returns the contentUrl of the first attachment with a contentType that starts with \"image\/\"\nconst inlineImageUrl = utils.getFirstInlineImageAttachmentUrl(session.message);\nif (inlineImageUrl) {\n    \/\/ Image was sent as inline content\n    \/\/ contentUrl is a url to the file content; the bot's access token is required\n    this.returnRecognizedTextAsync(session, async () =&gt; {\n        const buffer = await utils.getInlineAttachmentContentAsync(inlineImageUrl, session);\n        return await this.visionApi.runOcrAsync(buffer);\n    });\n    return;\n}<\/pre>\n<h2 id=\"toc-hId--1736967811\">Sending files<\/h2>\n<p>Similarly, your bot can send the user a file in 1:1 chat. There are several steps:<\/p>\n<h3 id=\"toc-hId--190670981\">1)\u00a0Request permission to upload the file<\/h3>\n<p>First, ask the user for permission to upload a file by sending a file consent card. For example:<\/p>\n<pre>{\n    \"contentType\": \"application\/vnd.microsoft.teams.card.file.consent\",\n    \"name\": \"result.txt\",\n    \"content\": {\n        \"description\": \"Text recognized from image\",\n        \"sizeInBytes\": 4348,\n        \"acceptContext\": {\n            \"resultId\": \"1a1e318d-8496-471b-9612-720ee4b1b592\"\n        },\n        \"declineContext\": {\n            \"resultId\": \"1a1e318d-8496-471b-9612-720ee4b1b592\"\n        }\n    }\n}<\/pre>\n<ul>\n<li><code>contentType<\/code>\u00a0is\u00a0<code>application\/vnd.microsoft.teams.card.file.consent<\/code>.<\/li>\n<li><code>name<\/code>\u00a0is the proposed file name.<\/li>\n<li><code>content.description<\/code>\u00a0is a description of the file.<\/li>\n<li><code>content.sizeInBytes<\/code>\u00a0is the approximate file size in bytes.<\/li>\n<li><code>content.acceptContext<\/code>\u00a0and\u00a0<code>content.declineContext<\/code>\u00a0are values that will be sent to the bot if the user accepts or declines, respectively.<\/li>\n<\/ul>\n<p>To help the user decide, we recommend providing name, description and size.<\/p>\n<h5 id=\"toc-hId-1159112344\">OCR Bot<\/h5>\n<p>The code for sending the file consent card is in the\u00a0<code>returnRecognizedTextAsync<\/code>\u00a0function. When we detect text in the image, first we save the result in conversation data, assigning it a randomly-generated result id. We\u2019ll need the text later, when the user accepts the file. We include the result id in the consent card\u2019s context to associate it with a specific OCR result.<\/p>\n<p>&nbsp;<\/p>\n<pre>\/\/ Save the OCR result in conversationData, while we wait for the user's consent to upload the file.\nconst resultId = uuidv4();\nsession.conversationData.ocrResult = {\n    resultId: resultId,\n    text: text,\n};\n\n\/\/ Calculate the file size in bytes. Note that this only needs to be approximate, \n\/\/ so if it's expensive to determine the file size, you don't need to do that.\n\/\/ In this case it's straightforward to get the actual size, so we might as well. \nconst buffer = new Buffer(text, \"utf8\");\nconst fileSizeInBytes = buffer.byteLength;\n\n\/\/ Build the file upload consent card\n\/\/ Accept and decline context contain the result id, to detect the case where the user acted on a stale card\nconst fileConsentCard = new msteams.FileConsentCard(session)\n    .name(filename || \"result.txt\")\n    .description(\"OCR result\")\n    .sizeInBytes(fileSizeInBytes)\n    .context({\n        resultId: resultId,\n    });\n\n\/\/ Send the text prompt and the file consent card.\n\/\/ We send them in 2 separate activities, to be sure that we can safely delete the file consent card alone.\nsession.send(\"I found text in %s\", result.language);\nsession.send(new builder.Message(session).addAttachment(fileConsentCard));<\/pre>\n<p>Note that the\u00a0<code>sizeInBytes<\/code>\u00a0value is just informational, and it can be approximate. In this case we already have the text, so it\u2019s straightforward to calculate the exact size. But if it\u2019s expensive to create the file, you can opt to include the necessary information in\u00a0<code>acceptContext<\/code>, and defer generating the content until\u00a0<em>after<\/em>\u00a0the user accepts the upload.<\/p>\n<h3 id=\"toc-hId--1000017607\">2)\u00a0User accepts or declines the file<\/h3>\n<p>When the user presses either \u201cAccept\u201d or \u201cDecline\u201d, your bot will receive an invoke activity.<\/p>\n<pre>{\n    \"type\": \"invoke\",\n    \"name\": \"fileConsent\/invoke\",\n    ...\n    \"value\": {\n        \"type\": \"fileUpload\",\n        \"action\": \"accept\",\n        \"context\": {\n            \"resultId\": \"82af7356-d2ef-4430-8a48-1ee189ca01db\"\n        },\n        \"uploadInfo\": {\n            \"contentUrl\": \"https:\/\/&lt;onedrive_url&gt;\/result.txt\",\n            \"name\": \"result.txt\",\n            \"uploadUrl\": \"https:\/\/&lt;onedrive_upload_url&gt;\",\n            \"uniqueId\": \"03AA04A0-4D33-4760-94C4-FEE498B68FF2\",\n            \"fileType\": \"txt\"\n        }\n    },\n    \"replyToId\": \"1:xxxxxxx\u201d\n}<\/pre>\n<ul>\n<li><code>name<\/code>\u00a0is always\u00a0<code>fileConsent\/invoke<\/code>.<\/li>\n<li><code>value.type<\/code>\u00a0is\u00a0<code>fileUpload<\/code>.<\/li>\n<li><code>value.action<\/code>\u00a0is\u00a0<code>accept<\/code>\u00a0if the user allowed the upload, or\u00a0<code>decline<\/code>\u00a0otherwise.<\/li>\n<li><code>value.context<\/code>\u00a0is either the\u00a0<code>acceptContext<\/code>\u00a0or the\u00a0<code>declineContext<\/code>\u00a0in the card, depending on whether the user accepted or declined.<\/li>\n<\/ul>\n<p>If the file was accepted, value contains an\u00a0<code>uploadInfo<\/code>\u00a0property with the following information:<\/p>\n<ul>\n<li><code>name<\/code>\u00a0is the name of the file. Note that this may be different from the name that the bot proposed initially.<\/li>\n<li><code>fileType<\/code>\u00a0is the file type, as determined by OneDrive.<\/li>\n<li><code>contentUrl<\/code>\u00a0is a direct link to the final location of the file on OneDrive.<\/li>\n<li><code>uploadUrl<\/code>\u00a0is the upload URL of the file. This points to a\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/onedrive\/developer\/rest-api\/api\/driveitem_createuploadsession\" target=\"_blank\" rel=\"noopener noreferrer\">OneDrive upload session<\/a>\u00a0for the file. The upload session is valid for 15 minutes.<\/li>\n<\/ul>\n<p>To set the file contents, issue PUT requests to the URL in\u00a0<code>uploadInfo.uploadUrl<\/code>, as described in the\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/onedrive\/developer\/rest-api\/api\/driveitem_createuploadsession#upload-bytes-to-the-upload-session\" target=\"_blank\" rel=\"noopener noreferrer\">documentation<\/a>. If your bot cannot complete the upload\u2014for example, if it hits an error while generating the file\u2014it\u2019s good practice to cancel the upload session by sending a DELETE request to the upload URL.<\/p>\n<p>Your bot has 10 seconds to respond to the invoke message. If it might take longer than that to generate and upload the file, acknowledge the invoke by returning a successful HTTP response (e.g., 202 Accepted), and then finish the upload asynchronously. Keep in mind that the user is likely waiting for the file, so remember to send updates!<\/p>\n<h3 id=\"toc-hId-742792728\">3)\u00a0Send the user a link to the uploaded file<\/h3>\n<p>After you finish the upload, we recommend sending a link to the file. A file info card works well, as the user can click on the card to open the file directly in Teams.<\/p>\n<pre>{\n    \"contentType\": \"application\/vnd.microsoft.teams.card.file.info\",\n    \"contentUrl\": \"&lt;uploadInfo.contentUrl&gt;\",\n    \"name\": \"&lt;uploadInfo.name&gt;\",\n    \"content\": {\n        \"uniqueId\": \"&lt;uploadInfo.uniqueId&gt;\",\n        \"fileType\": \"&lt;uploadInfo.fileType&gt;\",\n    }\n}<\/pre>\n<ul>\n<li><code>contentType<\/code>\u00a0is\u00a0<code>application\/vnd.microsoft.teams.card.file.info<\/code>.<\/li>\n<li>The other fields are taken from the\u00a0<code>uploadInfo<\/code>\u00a0value received with the\u00a0<code>fileConsent\/invoke<\/code>\u00a0message.<\/li>\n<\/ul>\n<h3 id=\"toc-hId--1809364233\">4)\u00a0(Optional) Delete or update the file consent card<\/h3>\n<p>To prevent the user from acting on a file consent card multiple times, you can delete or update the consent card. The\u00a0<code>replyToId<\/code>\u00a0property of the\u00a0<code>fileConsent\/invoke<\/code>\u00a0activity contains the activity id of the file consent card. Use this id in the corresponding APIs to\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/microsoftteams\/platform\/concepts\/bots\/bot-conversations\/bots-conversations#deleting-messages\" target=\"_blank\" rel=\"noopener noreferrer\">delete<\/a>\u00a0or\u00a0<a href=\"https:\/\/docs.microsoft.com\/en-us\/microsoftteams\/platform\/concepts\/bots\/bot-conversations\/bots-conversations#updating-messages\" target=\"_blank\" rel=\"noopener noreferrer\">update<\/a>\u00a0the activity.<\/p>\n<h5 id=\"toc-hId--459580908\">OCR Bot<\/h5>\n<p>The bot processes the file consent invoke activity in\u00a0<code>handleFileConsentResponseAsync<\/code>. First, it determines from\u00a0<code>value.action<\/code>\u00a0whether the user accepted or declined. If the user declined, we simply acknowledge the action, and delete both the file consent card and the OCR result. If the file was accepted:<\/p>\n<ol>\n<li>First, we check that the user acted on a consent card that corresponds to the current result. This helps guard against the user acting again on a previous card. In this case this is more of a precaution, because we proactively delete the consent card anyway.<\/li>\n<li>Then, we upload bytes into the file. It\u2019s important to follow the rules of the OneDrive upload session, otherwise you may receive errors. We have the full text ready, so we can upload the entire content in one go.<\/li>\n<li>If the upload succeeds, we send a file info card. The card is populated based on information in the\u00a0<code>uploadInfo<\/code>\u00a0object.<\/li>\n<\/ol>\n<pre>const lastOcrResult = session.conversationData.ocrResult;\n\n\/\/ Delete OCR result and file consent card\ndelete session.conversationData.ocrResult;\nconst addressOfSourceMessage: builder.IChatConnectorAddress = {\n    ...event.address,\n    id: event.replyToId,\n};\nsession.connector.delete(addressOfSourceMessage, (err) =&gt; {\n    if (err) {\n        console.error(`Failed to delete consent card: ${err.message}`, err);\n    }\n});\n\nconst value = (event as any).value as msteams.IFileConsentCardResponse;\nswitch (value.action) {\n    \/\/ User declined upload\n    case msteams.FileConsentCardAction.decline:\n        session.send(\"File upload declined\");\n        break;\n\n    \/\/ User accepted file\n    case msteams.FileConsentCardAction.accept:\n        const uploadInfo = value.uploadInfo;\n\n        \/\/ Check that this response is for the the current OCR result\n        if (!lastOcrResult || (lastOcrResult.resultId !== value.context.resultId)) {\n            session.send(\"Result has expired \");\n            return;\n        }\n\n        \/\/ Upload the file contents to the upload session we got from the invoke value\n        const buffer = new Buffer(lastOcrResult.text, \"utf8\");\n        const options: request.OptionsWithUrl = {\n            url: uploadInfo.uploadUrl,\n            body: buffer,\n            headers: {\n                \"Content-Type\": \"application\/octet-stream\",\n                \"Content-Range\": `bytes 0-${buffer.byteLength-1}\/${buffer.byteLength}`,\n            },\n        };\n        request.put(options, (err, res: http.IncomingMessage, body) =&gt; {\n            if (err) {\n                session.send(\"File upload error: %s, err.message);\n            } else if ((res.statusCode === 200) || (res.statusCode === 201)) {\n                \/\/ Send message with link to the file.\n                const fileInfoCard = msteams.FileInfoCard.fromFileUploadInfo(uploadInfo);\n                session.send(new builder.Message(session).addAttachment(fileInfoCard));\n            } else {\n                session.send(\"File upload error: %s\", res.statusMessage);\n            }\n        });\n        break;\n}<\/pre>\n<h2 id=\"toc-hId--1536920789\">Caution: developers at work\u00a0????<\/h2>\n<p>For now, the ability to receive file attachments and send files to the user is restricted to 1:1 chats with the bot. File attachments will continue to be dropped silently for messages sent to bots in channels and group chats. We know that working with files is a useful capability,\u00a0and we are working on extending this mechanism to those contexts as well.<\/p>\n<p>You might also run into some known issues. (We\u2019ll get these fixed before we make the feature generally available.)<\/p>\n<ul>\n<li>The file consent card shows only the file name, omitting the description and file size.<\/li>\n<li>If the bot sends a different file with the same name as one that already exists, the previous file is replaced. The user has no opportunity to select a new name.<\/li>\n<li>If the bot abandons an upload session for an\u00a0<em>existing<\/em>\u00a0file, then subsequent attempts to click on \u201cAccept\u201d could fail silently.<\/li>\n<\/ul>\n<p>Finally, note that\u00a0this is still in Public Developer Preview, and the exact API and UX\u00a0could change. That depends in part on feedback that we receive from our developer community\u2014yes, that\u2019s you! So please check out our\u00a0<a href=\"https:\/\/github.com\/aosolis\/visionsample-msteams-node\" target=\"_blank\" rel=\"noopener noreferrer\">Node.js<\/a>\u00a0and\u00a0<a href=\"https:\/\/github.com\/aosolis\/visionsample-msteams-csharp\" target=\"_blank\" rel=\"noopener noreferrer\">C#<\/a>\u00a0samples, try using the feature in your own app,\u00a0then let us know what you think. You can reach us through our various developer support channels:\u00a0<a href=\"https:\/\/github.com\/officedev\/botbuilder-microsoftteams\/issues\" target=\"_blank\" rel=\"noopener noreferrer\">Github<\/a>,\u00a0<a href=\"https:\/\/stackoverflow.com\/questions\/tagged\/microsoft-teams\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Stack Overflow<\/a>, and\u00a0<a href=\"mailto:microsoftteamsdev@microsoft.com\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">email<\/a>. We look forward to\u00a0hearing from you.<\/p>\n<p>Happy filing!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We announced at Build 2018 that bots will soon be able to send and receive files. I\u2019m happy to let you know that it\u2019s finally here! If you have a Microsoft Teams app that works with files, it\u2019s time to switch to Public Developer Preview and check it out.<\/p>\n","protected":false},"author":69074,"featured_media":25159,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[128],"tags":[37],"class_list":["post-1433","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft-teams","tag-bot-framework"],"acf":[],"blog_post_summary":"<p>We announced at Build 2018 that bots will soon be able to send and receive files. I\u2019m happy to let you know that it\u2019s finally here! If you have a Microsoft Teams app that works with files, it\u2019s time to switch to Public Developer Preview and check it out.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/1433","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/users\/69074"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/comments?post=1433"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/1433\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media\/25159"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media?parent=1433"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/categories?post=1433"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/tags?post=1433"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}