NSWI152 - Vývoj cloudových aplikací Filip Zavoral, Jaroslav Keznikl, Stanislav Ulrych http://ulita.ms.mff.cuni.cz/pub/predn/NSWI152/

Prezentace na téma: "NSWI152 - Vývoj cloudových aplikací Filip Zavoral, Jaroslav Keznikl, Stanislav Ulrych http://ulita.ms.mff.cuni.cz/pub/predn/NSWI152/"— Transkript prezentace:

1 NSWI152 - Vývoj cloudových aplikací Filip Zavoral, Jaroslav Keznikl, Stanislav Ulrych

2 Průběh semestru, platformy
0/2 Z, průběh semestru nepravidelný některé týdny cvičení - 'Guided Tour' v SW2 některé týdny praxe - samostatná práce Platformy Windows Azure Filip Zavoral 3 týdny cvičení - instalace, deployment, multi-tier aplikace 2 týdny praxe - vlastní aplikace OpenStack Stanislav Ulrych, Lukáš Korous 2 týdny cvičení + 2 týdny praxe Google App Engine Jaroslav Keznikl 3 týdny cvičení + 2 týdny praxe poslední seminář předvedení, prezentace, zhodnocení

3 Prerekvizity a požadavky
Znalosti NSWI150 Virtualizace a cloud computing speciálně část týkající se cloudů C#, ASP.Net, Python běžné věci na základní úrovni, copy-and-paste rozšiřující: NPRG057 Pokročilé programování pro .NET II Účty Microsoft Account (Windows Live ID) Google Account Virtuální stroje VMware vSphere Client / Web Client Windows 7, VS 2013, .Net 4.5

4 Windows Azure Filip Zavoral

5 Windows Azure Cloud Components
todo: mobile

6 Microsoft Azure

7 Studijní materiály, odkazy, harmonogram Příprava prostředí
Plán Studijní materiály, odkazy, harmonogram (Ulita  Výuka  NSWI152) Příprava prostředí Virtual Machines, RDP Instalace SDK, WATK Aktivace Windows Azure, Azure Management Portal Customer Management System Příprava cloudové infrastruktury Windows Azure Training Kit Vytvoření, deployment a spuštění aplikace GuestBook Vícevrstvá komplexnější reálná aplikace Web role, worker role, queues, blob, tables Vlastní aplikačka Individuálně nebo komplexnější aplikace ve dvojici Libovolný jazyk: .Net, Java, Php, Node.js, Python ...

8 Příprava prostředí a aktivace
Virtual Machines VMware vSphere Client bobr.ms.mff.cuni.cz login: cloud_xx  Grupíček pwd: v Grupíčku vSphere Web client [VCenter] VM: BOBR / VIRTLAB / Cloud / Students / cloud_xx login: administrator pwd: v Grupíčku nastavena IP xx  ověřit! subnet , GW , DNS /77 RDP enable RDP: Start / Computer / Properties / Remote Settings / Allow Power Options - never sleep acheron.ms.mff.cuni.cz:102xx aktualizace! - povolit auto, provést Aktivace Windows Azure Window Azure Management Portal Po redeemovani se musi cekat na mail (pry 2-3 dni, mne to prislo za chvili) Pred semestrem v labu instalace VMware VSphere klienta!!! Have a code Redeem

9 Microsoft Azure Academic Licence
Cloud Services 2 Small Instances for Cloud Services or Virtual Machines 10 Shared Web Sites, 10 Shared Mobile Services Data Management Two 1 GB Web Edition SQL Databases 35 GB of Storage 50,000,000 Storage Transactions Data Transfers 16 GB Bandwidth (8 GB In, 8 GB Out) Messaging 750 Service Bus Relay Hours 250,000 Service Bus Messages Identity 100,000 Access Control Transactions

10 Web Platform Installer - Azure SDK, WATK
Windows Azure SDK 2.2 pro VS 2013 Windows Azure Training Kit  c:\watk\HOL - hands-on labs

11 Před 2. cvičením mít zprovozněno:
Checklist Před 2. cvičením mít zprovozněno: virtual machine, RDP přes acheron W7 + aktualizace VS Windows Azure SDK 2.2 Windows Azure Training Kit (WATK) Microsoft Account (Windows Live ID) Aktivace Windows Azure - redeem code login to Azure Management Portal v případě jakýchkoliv problémů se včas (do pátku) přihlaste platí i pro ty, kteří nebyli na prvním cvičení! Studijní materiály Building Real-World Cloud Apps with Windows Azure leden 2014 Azure documentation, články, linky, ...

12 Customer Management System

13 Customer Management System
Windows Azure Training Kit Building and Publishing ASP.NET Applications with Windows Azure Web Sites watk\HOL\ASPNETAzureWebSites Customer Management System jednoduchý adresníček Objectives Create a Web Site from the Windows Azure Management Portal Use MS VS 2013 to build a new ASP.NET MVC 5 application Deploy the application using Web Deploy from Visual Studio Exercises E1: Creating a Windows Azure Web Site E2: Creating an MVC 5 Application with Entity Framework E3: Publishing an MVC 5 Application using Web Deploy (opt. E4: publish an ASP.NET MVC 5 application using Git or ftp) Adresnicek 

14 E1: Creating a Windows Azure Web Site (with Visual Studio)
Create Web Site E1: Creating a Windows Azure Web Site (with Visual Studio) Connect to Azure [1-3] View / Server Explorer / Windows Azure / [RMB] Connect to WA / … login  Create Site – se studentskými licencemi nefunkční! [4] SE / Windows Azure / Web Sites / Add New Site [5-6] Site name (unique! URL), any location, Database: Create new server DB admin login & pwd, Create – unable to create…  Create Site Using Azure Management Portal Azure Management Portal – dole ✚ / Compute / Website / Custom Create – URL, region, SQL DB, connection string  DB name, new, login, pwd  Create check: Web Sites, SQL Databases Back to WATK [7] Portal: explore Web Sites / mywebsite / Scale (do not change anything) [8] VS/SE: Refresh, Web Sites! (SQL Databases still empty) mywebsite / Open in Browser Zapište si!

15 E2: Creating an MVC 5 Application with Entity Framework
Create Application E2: Creating an MVC 5 Application with Entity Framework Create New Project [1-2] Installed Templates, Visual C# Web, .NetFramework 4.5, ASP.Net Web Application [3] MVC, check: Authentication: Individual User Accounts Model-View-Controller Solution explorer: Models, Views, Controllers, ..., ... Create Customer Model [4] Solution Explorer/Models: Add class Customer [5-6] Models/Customer.cs: add properties, build Create Person Controller & View [7-8] Solution Explorer/Controllers, Add Controller, MVC5 using EntFwk, Add [9] vyplnit přesně dle WATK use async controller, Customer model [10] Add actions, data context & views are generated [11] Controllers/CustomersController.cs: see generated methods See async, await – asynchroneous execution Po [3] se zobrazi readme a spousta linku, ignore MVC – predgenerovane .cs [9] vyplnit options podle labu!!

16 E3: Publishing an MVC 5 Application using Web Deploy
Publish Application E3: Publishing an MVC 5 Application using Web Deploy Establish Connection [1-2] SE / MVCSample.Web / [RMB] Publish [3-5] Publish Web: Profile ⊳ Import  Import from Azure / mywebsite / OK Dialog is filled-in with the imported connection data [6] Connection ⊳ Validate Connection ☑ / Next Connect to Database [7] Settings ⊳ CustomerContext / mydatabase / Next Publish [8] Preview ⊳ Publish Build started … [chvíli trvá] ... Site was published successfully [9] Check the browser, check url Use the Application [10] url: /Customer Play the game – new, edit, delete Check the source code

17 E4: Publishing an MVC 5 Application using Git
Git & ftp Access Samostatně E4: Publishing an MVC 5 Application using Git Dle WATK E5: Ftp Access Portal / WebSites / mywebsite / Dashboard / Quick Glance Set Deployment Credentials ftp/ftps host name, ftp user (incl. prefix!) connect, browse

18 GuestBook

19 Windows Azure Training Kit Objectives
GuestBook Windows Azure Training Kit HOL\IntroToCloudServicesVS2012 Objectives Create applications in Windows Azure using web roles and worker roles Use Storage services including blobs, queues and key/value tables Publish an application to Windows Azure Cloud Services Setup IntroToCloudServicesVS2012\Source\setup.cmd - Run as administrator ! nestahovat VC2012 ! web role ≠ web site cloud application != web site

20 E1: Azure Application with Blobs and Tables
GuestBook Outline E1: Azure Application with Blobs and Tables Task 1 - Creating the VS Project Task 2 - Creating a Data Model for Table Storage Task 3 - Creating a Web Role to Display the Guest Book and Process User Input Task 4 - Queuing Work Items for Background Processing E2: Background Processing with Worker Roles and Queues Task 1 - Creating a Worker Role to Process Images in the Background E3: Publishing a Windows Azure Application Task 1 - Creating a Storage Account and a Cloud Service Component Task 2 - Publishing the Application to the Azure Portal Task 3 - Configuring the Application to Increase the Number of Instances Task 4 - Testing the Application in the Staging Environment Task 5 - Promoting the Application to Production

21 E1T1 - Create Application
E1: Building Windows Azure Application with Blobs and Tables Task 1 - Creating the Visual Studio Project Create Cloud Service Project [1-2] Run VS 2013 as administrator Windows Azure Compute Emulator [3-4] Installed Templates, Visual C# Cloud, .Net 4.5, W Azure Cloud Service, vyplnit Name: GuestBook, Solution name: Begin Create Web Service Web Role [5] New W Azure Cloud Service, Visual C#, ASP.Net Web Role ⍄ GuestBook_WebRole MVC [6] Solution Explorer - explore Projects in Solution GuestBook - service definition, local / production configuration of roles, .cscfg GuestBook_WebRole - ASP.Net Web Application GuestBook_WorkerRole, GuestBook_Data, later Task 2 - Creating a Data Model for Entities in Table Storage Task 3 - Creating a Web Role to Display the Guest Book and Process User Input Task 4 - Queuing Work Items for Background Processing po 5 - novy (VS2013) dialog! zvolit MVC

22 E1T2 - Data Model Task 2 - Creating a Data Model for Entities in Table Storage Key/Value Tables Key/Value Table - scalable data storage Table storage API: WCF Data Services (ADO.Net Data Services Framework) - REST API WCF Data Services Client Library (.Net Client Library) - direct access from C# objects MyEntity : TableServiceEntity - schema class MyContext : TableServiceContext - table access MyContext.IQueryable<MyEntity> - table definition and mapping Create Data Project [1-2] New Project, Installed, Visual C#, Windows, .Net 4.5, ClassLibrary, GuestBook_Data Solution: Add to Solution ! [3] delete Class1.cs [4-5] Solution explorer, GuestBook_Data, References, Add Reference Framework ⊳ System.Data.ClientServices, Extensions ⊳ Microsoft.WindowsAzure.Storage, Microsoft.WindowsAzure.Configuration, Create Schema Class [6] SE: GuestBook_Data, Add, New Item ⊳ Visual C# Items, Class, GuestBookEntry.cs [7] inherit GuestBookEntry class from TableServiceEntity TSE defines PartitionKey, RowKey, TimeStamp nove / (stare) nazvy - prejmenovali DS poskytuje REST API DSCL umoznuje primo pracovat s C# objekty vytvorime MyEntity a MyContext - odvozene od .... reference - nejnovejsi verze (2.1, 2.0) (old [7] import StorageClient library) - to uz se nedela, referencuje se primo Storage

23 E1T2 - Data Model Task 2 - Creating a Data Model ... continued
Define Schema PartitionKey  day - partitioning, RowKey  reverse time - sort order [8] default constructor - initializes keys [9-10] other properties - data row , save Message, GuestName, PhotoUrl, ThumbnailUrl Create Context Class [11] GuestBook_Data, Add, New Item ⊳ Class, GuestBookDataContext.cs [12-14] GuestBookDataContext, constructor, GuestBookEntry property [15-16] Add Class GuestBookDataSource.cs - object bound to data controls [17-18] add storageAccount, context, constructor [19-23] finalize bindings, queries and initializations, save TableQuery<GuestBookEntry> query = new TableQuery<GuestBookEntry>() .Where( TableQuery.GenerateFilterCondition( "PartitionKey", QueryComparisons.Equal, DateTime.UtcNow.ToString("MMddyyyy"))); return table.ExecuteQuery(query); 2013: Builder GetGuestBookEntries UpdateImageThumbnail The static constructor initializes the storage account by reading its settings from the configuration and then uses the CreateTablesFromModel method in the CloudTableClient class to create the tables used by the application from the model defined by the GuestBookDataContext class. By using the static constructor, you ensure that this initialization task is executed only once. tyhle dotazy byly vloni v integrovanem LINQu, letos to jsou C# builder-metody zajimavy, pred tim se tim tak chlubili :-) results = from g in this.context.GuestBookEntry where g.PartitionKey == DateTime.UtcNow.ToString("MMddyyyy") select g; 2012: LINQ

24 E1T3 - Web Role Task 3 - Creating a Web Role to Display the Guest Book and Process User Input Create Application Page [1] SE: GB_WebRole, Add reference to Solution / GuestBook_Data [2] delete automatically generated default page GBWR/Default.aspx [3] Add existing item watk\....\Source\Ex1...\Assets\*.* page already prepared by watk - code, images, styles - explore Process Guest Entry [4-6] GB_WebRole: Default.aspx.cs - add using, declarations note: using GuestBook_Data, CloudBlobClient [7] add event handler to SignButton_Click - upload to blob InitializeStorage (impl. later) gen unique blob name, create blob, upload new table entry save data using GBDataSource - To process a new guest book entry, the handler first calls the InitializeStorage method to ensure that the blob container used to store images exists and allows public access. You will implement this method shortly. - It then obtains a reference to the blob container, generates a unique name and creates a new blob, and then uploads the image submitted by the user into this blob. Notice that the method initializes the ContentType property of the blob from the content type of the file submitted by the user. When the guest book page reads the blob back from storage, the response returns this content type, allowing a page to display the image contained in the blob simply by referring to its URL. - After that, it creates a new GuestBookEntry entity, which is the entity you defined in the previous task, initializes it with the information submitted by the user, and then uses the GuestBookDataSource class to save the entry to table storage. - Finally, it data binds the guest book entries list to refresh its contents. GuestBookEntry entry = new GuestBookEntry() { GuestName = this.NameTextBox.Text, Message = this.MessageTextBox.Text, PhotoUrl = blob.Uri.ToString(), ThumbnailUrl = blob.Uri.ToString() }; GuestBookDataSource ds = new GuestBookDataSource(); ds.AddGuestBookEntry(entry);

25 E1T3 - Web Role Task 3 - Creating a Web Role ... continued
Load Page - Timer [8] Timer1_Tick - periodical refresh [9] Page_Load - enable timer Initialize Storage [10] InitializeStorage execute only once, read account settings, create and configure blob container Create Storage [11] SE: GB, Roles, GBWR [RTMB] Properties Settings ⊳ Add Setting "DataConnectionString" / ConnectionString [...] [12-13] Connect using WAzure Storage Emulator, save emulator v ramci VS SDK, pro deployment se dosadi spravny connection string - nezapomenout! storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString"); blobStorage = storageAccount.CreateCloudBlobClient(); container = blobStorage.GetContainerReference("guestbookpics"); container.CreateIfNotExist();

26 E1T4 - Queue & Run Task 4 - Queuing Work Items for Background Processing Declare and Initialize Queue [1-2] Default.aspx.cs, class _Default - declare queueStorage [3] InitializeStorage - add initialization code after blob queue "guestthumbs" Enqueue Message [4] SignButton_Click - locate queue, form message, enqueue - add code Verification Run Application Windows Azure Compute Emulator (Development Fabric), Storage Browser check: Run VS 2013 as administrator Update to .Net Web.config - do sekce <appSettings> [1-3] F5 build & run - browser ➠ /Default.aspx system tray - Show Compute Emulator, firewall - allow access Play the game Storage Explorer [4-9] VS: View/ServerExplorer/Azure/Storage/Development - table, blob, queue na webu <add key="ValidationSettings:UnobtrusiveValidationMode" value="None" /> WACE - old name: development fabric data se strkaji do talbles, queues, blobs WebForms UnobtrusiveValidationMode requires nevtíravý :-) server explorer - ve watk je db explorer

27 E2: Background Processing with Worker Roles and Queues
E2 - Worker Role E2: Background Processing with Worker Roles and Queues Task 1 - Creating a Worker Role to Process Images in the Background Create Worker Role [1-2] stop debugging, continue with the solution [3-4] GuestBook, Roles, New Worker Role - GuestBook_WorkerRole [5-6] add references - GuestBook_Data, System.Drawing Programming Worker Role - Create Thumbnails [7-8] WorkerRole.cs: add using, WorkerRole: add queue and blob properties [9] OnStart - worker initialization read config, init blob & queue, catch [10] Run - replace default body - main worker logic fetch & parse msg, read blob, create thumbnail, write blob, update thumbnail using - unmanaged IDisposable objects processing [11] ProcessImage - create thumbnail, save to output stream Storage Service [12] Create WAzure Storage Emulator ( E1T ) Verification [1-6] Run - wait a moment, create an entry, wait a moment [4] Server Explorer - explore data [9] catch - for debugging old images

28 E3T1 - Create Storage and Component
E3: Publishing a Windows Azure Application Task 1 - Creating a Storage Account and a Cloud Service Component Create Virtual Network [1-4] Portal / ✚ / Networks / Virtual Network / Custom Create Network Name, Affinity Group Name [5-7]  - no other options! Create Cloud Service [8-9] ✚ / Compute / Cloud Service / Quick Create unique URL Create Storage [10-11] ✚ / Data Services / Storage / Quick Create the same URL as [9] (if possible) [12-14] Manage Keys - Primary Access Key - copy to clipboard [15] do not close the browser

29 E3T2 - Publish Task 2 - Publishing the Application to the Azure Portal
Deployment Options Visual Studio / PowerShell WAzure cmdlets / WAzure Portal Service Configuration [1-3] continue with the solution (or Ex3/.../GuestBook.sln) [4-7] SE: GuestBook / Roles / GuestBook_WebRole / Properties Settings ⊳ DataConnectionString [...] Manually entered [8] Microsoft.WindowsAzure.Plugins.... totéž GuestBook_WebRole - totéž (oba klíče) Generate Package [9] VS: GuestBook [RMB] Package: Cloud/Debug enable NuGet download ...\app.publish\*.cspkg, .cscfg Upload Package [11-19] Cloud Service, Dashboard, Staging, Upload [16] !! check Deploy even if one or more roles contain a single instance - wait Play the game, check Portal, ..., ... Task 3 - Configuring the Application to Increase the Number of Instances Task 4 - Testing the Application in the Staging Environment Task 5 - Promoting the Application to Production při překliknutí přes Your subscription se správně vyplní otevře se folder s package nejdřív vyzkoušet v staging, potom upgrade to production [9b] watk note - jak publishovat primo z VS NuGet - kdyz to bude VS chtit

30 Useful tools & practices
Storage explorer V4 - stable Useful Labs Mobile AndroidAndMobileServices HTMLAppAndMobileServices MobileServicesCustomAPI Other services IntroToWindowsAzureMediaServices ServiceBusTopics DebuggingCloudServices Practices Azure Blogs Using Trace azure-cloud-applications-1.aspx .cscfg: Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString

31 Cloud Development Patterns
Automate everything Use scripts to maximize efficiency and minimize errors in repetitive processes Source control Set up branching structure in source control Continuous integration and delivery Automate build and deployment with each source control check-in Web development best practices Keep web tier stateless; Avoid session state; Use a CDN; Use asynchronous programming model Data storage options Types of data stores; How to choose the right data store; Unstructured blob storage Data partitioning strategies Partition data vertically, horizontally, or both to facilitate scaling a relational database Design to survive failures Types of failures; Failure Scope; Understanding SLAs Monitoring and telemetry Why you should both buy a telemetry app and write your own code to instrument your app Transient fault handling Use smart retry/back-off logic to mitigate the effect of transient failures Distributed caching Improve scalability and reduce database transaction costs by using distributed caching Queue-centric work pattern Enable high availability and improve scalability by loosely coupling web and worker tiers Single sign-on - Active Directory Building Real-World Cloud Apps with Windows Azure the book

32 Dokončení a předvedení
Vlastní aplikace Vlastní aplikace individuálně nebo větší aplikace ve dvojici cokoliv, co využije větší počet modulů a služeb Web-Worker Roles, Key/Value Tables, BLOBs, Mobile, Media, Service Bus, Hadoop, .... vloni - pro inspiraci Mobilní kurýr s balíčkem, Pivar, Pet gallery, Azgool Plus, Twitter, Vysledky florbalu, Jídelníčky menz pro mobily, Konverze a filtr velkých XML, Renderovani map, Bugzilla, Cestovní agentura, Rapidshare / Ulozto, Kontroly revizorů Dokončení a předvedení poslední cvičení 20.5. slajdy - prezentace co to dělá jak je to udělané moduly a služby, vazby mezi nimi zhodnocení - co bylo zajímavé, kladně i záporně srovnání s ostatnímiplatformami ukázka reálného běhu

33 OpenStack Stanislav Ulrych / Lukáš Korous

34 Google App Engine Jaroslav Keznikl

35 The End. the book