WHAT'S NEW?
Loading...
Showing posts with label Web service. Show all posts
Showing posts with label Web service. Show all posts

Index


  • Introduction
  • XML serializer
  • Binary serializer
  • DataContract serializer
  • JSON serializer
  • References

Introduction

This post is part of a series of post to help you prepare the MCSD certification, particularly the certification exam 70-483, based on the book:


You will find all the code available in the following GitHub repository. Lets talk a little bit about threads and how they work using a .Net development environment.

XML serializer

Serialization is about transformation in order to send information. When you want to send data to a web service first you transform that data to a flat or binary form, send it and then you transform back that data. When you want to define what data you want to serialize, typically a DTO (Data transfer object) is created where you define all the properties you want to send.

Features:
  • XmlSerializer class
  • Simple Object Access Protocol (SOAP)
  • Not the best performance
  • Can't do private fields
  • Decoration:
    • class: [Serializable]
    • properties: by default members are serialized as XmlElement, ie, as nodes.
      • [XmlIgnore]: not serialize
      • [XmlAttribute]
      • [XmlElement]
      • [XmlArray]: for collections
      • [XmlArrayItem]: for collections

Binary serializer

This type of serializer is used more commonly with images and non-readable data. 

Features:
  • BinarySerializer class
  • namespaces: System.Runtime.Serialization and System.Runtime.Serialization.Formatters.Binary
  • Private fields are serialized by default
  • Decoration:
    • class: [Serializable]
    • properties:
      • [NonSerialized]
    • methods:
      • [OnDeserializedAttribute]
      • [OnDeserializingAttribute]
      • [OnSerializedAttribute]
      • [OnSerializingAttribute]
  • ISerializable: by implementing this you can define what is serialized, and if you have sensitive data you can encrypt prior to serialization. Implement:
    • GetObjectData(): called when your object is serialized. You can decorate it with a SecurityPermission attribute, so it is allowed to serialize and deserialize
    • protected constructor: called when you object is deserialized. You use it to retrive the values and build your object by defining 

DataContract serializer

Features:
  • DataContractSerializer class
  • It's used by WCF to serialize to XML or JSON
  • Decoration:
    • class: [DataContract]
    • operations: [DataMember]
    • methods:
      • [OnDeserializedAttribute]
      • [OnDeserializingAttribute]
      • [OnSerializedAttribute]
      • [OnSerializingAttribute]

JSON serializer

Features:
  • DataContractJsonSerializer class
  • small amounts of data
  • It uses Asynchronous JavaScript and XML (AJAX)

Index


  • Introduction
  • Databases
  • XML
  • JSON
  • Web services
  • References

Introduction

This post is part of a series of post to help you prepare the MCSD certification, particularly the certification exam 70-483, based on the book:


You will find all the code available in the following GitHub repository. Lets talk a little bit about threads and how they work using a .Net development environment.

Databases


Applications need a persist storage system to save data, this can be done in a database using ADO.NET or the Entity Framework. You can also use a web service and retrieve a response in JavaScript Object Notation (JSON) or Extensible Markup Language (XML). All the .Net functionalities to work with databases is stored within the System.Data namespace and it consists on two different approaches: 
  • connected: by running queries by using Structured Query Language (SQL) to create, read, update, and delete data (known as CRUD operations)
  • or disconnected data: you'll use DataSets and DataTables to mimic the database structures. Any change made can be sent back to the data store by using a DataReader.
Providers:
  1. Microsoft
  2. SQL
  3. Oracle
  4. MySQL
Connecting: we always need to define some connection (DbConnection class) details using a connectionString and providing the database type, the location and the credentials to log in. Connections are used within a using statement, IDisposable is implemented. Typically this strings are stored in a config file (app.config or web.config) but you can build your dynamically by using the DbConnectionStringBuilder class: OracleConnectionStringBuilder, SQLConnectionStringBuilder... Hard-code this is a bad practice, as mentioned before you can save them in a config file and retrieve them by calling the ConfigurationManager class like here:


Connecting is a time consuming operation and leave a connection open can prevent other users to access the storage, instead you can use the connection pool to save some resources and time.

Selection: when running a particular query against a database you can use the SqlCommand class which returns a SqlDataReader which keeps track of where you are in the result set. Asyn/Await is supported as well. SqlDataReader is a forward-only stream. You can't go back while you're reading, you can access the columns by index and by name by calling:

  • GetInt32(int index)
  • GetGuid(int index)
  • GetString(int index)
You can even batch multiple operations together and in result the SqlDataReader will return multiple result sets. Then you can move over them by calling NextResult() or NextResultAsync().

Update: here when you change something in a database you don't get a set with the information affected by your query, instead you end up with an integer which represents the amount of elements that have been modified. Run a ExecuteNonQuery() or ExecuteNonQueryAsync() in a command.

Parameters: typically you use parameters in your queries when filtering your selections or when updating data. Never hook your user interface components with you queries as this is a potential SQL injection attack candidate. Instead, use parameterized SQL which produces a more generic query and is easier to precompile an execution plan producing a more secure and better performance result.

Transactions: (ACID) key properties:
  1. Atomicity: if one fails, they all fail (rollback).
  2. Consistency: from one valid state to another.
  3. Isolation: Multiple concurrent transactions won't influence each other.
  4. Durability: committed transactions result is always stored permanently.
If nothing goes wrong you call TransactionScope.Complete() within a using statement. Transactions can be created using three options:

  1. Required: Join the ambient transaction or create a new one if it doesn't exist.
  2. RequiresNew: Start a new transaction.
  3. Suppress: Don't take part in any transaction.
The .Net framework manages transactions for you. If the transaction uses nested connections, multiple databases or multiple resources it will be promoted to a distributed transaction (avoid if possible).

ORM (Object Relational Mapper): you can manually write your SQL statements but if your app grows it end up being a nightmare to maintain or improve. Here is when ORMs like Entity Framework come handy generating for you all those queries. 

It provides three different approaches:
  1. Database First
  2. Model First: typically use a graphical tool
  3. Code First: you need to define all in code, your entities, relations and so... inherit 
    1. DbContext is used to create your own context, which is the interface between your code and the database. 
    2. By adding entities to the context you'll end up creating new rows and to commit that calling the SaveChanges() method. 
    3. Conventions are applied like the Id property is a primary key and more...

XML

It's a document formatted to be readable both by humans and computers. First line is an optional line which is called prolog and tells you are reading to an xml file and the enconding used, it typically looks like this:

<?xml version="1.0" encoding="UTF-8" ?>

Now we'll take a look at some .Net classes to help us work with this type of files:

  • XmlReader: read xml files in hierarchical manner and only forward without cached
    • Create(): static. Param XmlReaderSettings to configure how to read and skip some data.
  • XmlWriter: write xml files only forward without cached.
    • Create(): static. Param XmlWritterSettings.
  • XmlDocument: navigate and edit for smaller documents as it's slower than XmlReader/Writer. After editing you can save. It uses XmlNode to move through your document and perform changes of attributes in nodes. A nifty way to navigate is by using XPath (query language for xml). XPathNvigator class offers an easy way to navigate through an XML document.
  • XPathNavigator: navigating through an XML
See an example here of how XML looks like:

<employees>
    <employee>
        <firstName>John</firstName> <lastName>Doe</lastName>
    </employee>
    <employee>
        <firstName>Anna</firstName> <lastName>Smith</lastName>
    </employee>
    <employee>
        <firstName>Peter</firstName> <lastName>Jones</lastName>
    </employee>
</employees>

JSON

Lighter version of XML is JSON or JavaScript Object Notation, this contains less rules and therefor is why is light. The .Net library Newtonsoft.Json available at http://json.codeplex.com/ offers some functionalities to work with this format. Typically is used with asynchronous calls between web sites and servers or alson known as AJAX (Asynchronous JavaScript and XML, in reality XML was replaced by JSON so AJAJ is more accurate).

See here below the previous XML example this time using JSON

{"employees":[
    {"firstName":"John""lastName":"Doe"},
    {"firstName":"Anna""lastName":"Smith"},
    {"firstName":"Peter""lastName":"Jones"}
]}


Web services

Web services and micro web services turned in loosely coupled solutions for those who want to integrate to splitted systems. .Net offers the Windows Communication Framework (WCF) to build you own services. You will build a class with all the logic you service offers and it will be decorated with the attribute: ServiceContract close to the class name. Bear in mind you have to add System.ServiceModel reference into your project. For each of the methods exposed you will use the OperationContract decorator. Example:



Every WCF service has the following ABC properties:

  • Address: defines the endpoint. This is the URL you have to tackle to open the connection.
  • Bindings: it configures the protocols and transports that can be used to call your service: HTTP, HTTPS, named-pipe connections...
  • Contract: this defines which are the operations your web service exposes.
If the web service is already created, VS offers you a handy way to add a reference into your project which will help you to map all the methods exposed. Behind the scenes, the proxy uses the configuration file (app.config, web.config) to get the ABC settings.

Index

  • Intro
  • File System
  • Modify files
  • What are web services
  • REST Services
  • Asynchronous programming

Intro

I want to talk today about how to work with the file system and why we need it, so let's talk about the different scenarios where you'll be using the System.IO .Net library.


Besides we'll be scratching the surface of a couple of web services called "SOAP" and "REST", which you could be already familiar with. Finally, asynchronous programming will come over.

File System

Benefits of read and write from the file system:

  • Show existing data to user
  • Integrate user-provided data
  • Serialize objects out of memory
  • Persist data across sessions
  • Determine environment configuration

How to write:

  • This is simplified with Framework methods; 
    • open / shut (File.WriteAllText / File.ReadAllText) (see example below)
  • Open for reading to keep open and keep writing.
  • Open as stream for large payloads and real-time processing: let's say you want to see a movie through Netflix, you don't download the entire movie file into your PC and then you start watching. You download just a small peace of that huge file depending on where you are in the movie.
See an example of how write/read a file using .Net


var dir = System.IO.Directory.GetCurrentDirectory();
var file = System.IO.Path.Combine(dir, "File.txt");
var content = "how now brown cow?";

// write
System.IO.File.WriteAllText(file, content);

// read
var read = System.IO.File.ReadAllText(file);
Trace.Assert(read.Equals(content));


How do we find files?

  • Get Windows folder with Environment Special Folder
  • Get the Current folder with File.IO.GetCurrentDirectory()
  • Use Isolated Storage dedicated to the current application: 
    • dedicated application storage not used very often by developers.
  • Anything Else.Caveat: Windows Store App development

// special folders
var docs = Environment.SpecialFolder.MyDocuments;
var app = Environment.SpecialFolder.CommonApplicationData;
var prog = Environment.SpecialFolder.ProgramFiles;
var desk = Environment.SpecialFolder.Desktop;

// application folder
var dir = System.IO.Directory.GetCurrentDirectory();

// isolated storage folder(s)
var iso = IsolatedStorageFile
    .GetStore(IsolatedStorageScope.Assembly, "Demo")
    .GetDirectoryNames("*");

// manual path
var temp = new System.IO.DirectoryInfo("c:\temp");

In the previous example you can see how to reach the typical windows directories, your app and your isolated storage folder in .Net. Finally directoryInfo provides you useful info about certain directory like size, rights, etc. You are not allowed to do whatever you want in your PC, at some folder you probably need some rights in order to read, write or... delete!

Modify files

Please see here a small piece of code to show you how to edit files, following these steps:

  • Iterate through files using GetFiles()
  • Rename / Move with System.IO methods
  • Get File Info with System.UI.FileInfo

// files
foreach (var item in System.IO.Directory.GetFiles(dir))
    Console.WriteLine(System.IO.Path.GetFileName(item));

// rename / move
var path1 = "c:\temp\file1.txt";
var path2 = "c:\temp\file2.txt";
System.IO.File.Move(path1, path2);

// file info
var info = new System.IO.FileInfo(path1);
Console.WriteLine("{0}kb", info.Length / 1000);


See how we get all the files defined in the "dir" variable (let say "C:\temp") using GetFiles. Then we print all those files placed in the "dir" directory using GetFileName. Then moving a file to the same folder which purely it's a renaming operation. Finally, we'll get some meta data using the FileInfo method and we use the result of that method to print on the console the length of that file.

What are web services

See some web services (WS) features:

  • Web services encapsulate implementation: this means we don't know how a third party service is implemented. You can be using Facebook web services but you don't know how their WS are implemented.
  • WS expose to disparate system.
  • WS allow client systems to communicate servers: which is great because allow you to separate the different parts when you're building a new application.
    • Web protocols (HTTP, GET, POST, etc)
  • WS are important to Service Oriented Architecture
    • With and without metadata
    • Loose coupling: when a system has different moving parts depending on others and add something new means add a lot of extra changes.

What is SOAP? Easy, it's a type of web service. See some features:


  • SOAP is a standard for returning structured data from a Web Service as XML
    • Envelope (Header / Body / ...)
  • SOAP handling is a built-in feature of Visual Studio
The idea is to serialize the information provided by our application into our web service to send that information to our clients. Then our clients will de serialize that information in order to be able to read it.

The downside of this is the de/serialization process is heavy. At first, save 400 bytes it's not a big deal, but in case of companies like Facebook which could face millions of requests per day means a lot. There is a few emerging new technologies to bypass this problem...

REST Services

REST (Representative State Transfer)

  • ...is becoming a common, industry standard
  • ...does not require XML parsing
  • ...does not require message header
  • ...is generally human-readable
  • ...uses less band-with than SOAP
  • ...services typically return XML or JSON
  • ...JSON is JavaScript Object notation
    • JSON is becoming a common industry standard
    • JSON is generally a lighter payload than XML (or SOAP)

See how to implement a basic JSON reader using the .Net framework:

using System;
using System.Collections.Generic;
using System.Net;
using System.Web.Script.Serialization;

namespace ConsoleApplication1
{
    public class Program
    {
        public static void Main()
        {
            var url = new Uri("http://footballpool.dataaccess.eu/data/info.wso/TopGoalScorers/JSON/debug?iTopN=10");
            var client = new WebClient();
            var json = client.DownloadString(url);

            // deserialize JSON into objects
            var serializer = new JavaScriptSerializer();
            var data = serializer.Deserialize>(json);

            // use the objects
            foreach (var item in data)
                Console.WriteLine("Player: {0,16}, Goals: {1}", item.sName, item.iGoals);

            Console.Read();

            // Output
            // Player:  James Rodríguez, Goals: 6
            // Player:   Thomas Mueller, Goals: 5
            // Player:            Messi, Goals: 4
            // Player:           Neymar, Goals: 4
            // Player:       van Persie, Goals: 4
            // Player:   Andre Schurrle, Goals: 3
            // Player:          Benzema, Goals: 3
            // Player:   Enner Valencia, Goals: 3
            // Player:    Islam Slimani, Goals: 3
            // Player:           Robben, Goals: 3
        }
    }

    public class ObjectReturnedByTheWebService
    {
        public string sName { get; set; }
        public int iGoals { get; set; }

    }
}


If you want to try this solution, you could create a new console application and then add a new reference to System.Web.Extensions in order to get the JavaScriptSerializer class working. The URL I'm using it's a free web service with free football stats called: http://footballpool.dataaccess.eu/. I'm passing the value 10 when building the URL, but you can change that in order to get top3, top5...

I use the WebClient class to call the service and then I de-serialize the message because it comes as a JSON message. Try to go to that URL to see how it looks. There is a small class called ObjectReturnedByTheWebService which is used to convert that json message from plain text into a proper .Net class using a auto mapper.


As an example of how using REST services, please see how to implement an image recognition system in one of my first posts for this blog.

Asynchronous programming

Asynchronous maximizes resources on multi core systems, by allowing units of work to be separated and completed. Asynchronous programming frees up the calling system, especially a user interface, as to not wait for long operations. Don't using this approach will constrain your resources. It improves the user experience using your application by freezing the screen giving a feel like your app is hanging in a long running task.

References

Index


  1. Web Page construction
  2. Understanding ASP.NET
  3. Hosting Web sites
  4. Intro Web services

1. Web Page construction


The origin of web pages was to store documents and move from one to another easily. Over time, Internet has evolved and now you are able to find more functionalities and different services but all of these web pages are built with three main languages you should know:

  • uses HTML: Hypertext Markup Language, here the developer includes the structure of our documents. It defines the position of every element you can see in a web page.
  • uses CSS: responsible of the look of any page. Here we can talk about inheritance between the different groups defined in our html page that's why it takes the name of Cascade Style Sheet.
  • uses JavaScript (JS): gives functionality to our web page on the client side (because sometimes we don't want to go back to the server for "small things"), by creating links to other resources, user messages, data validation and a lot of new and real time functions if you are using frameworks like jQuery for instance.

All this information (HTML + CSS + JS) is interpreted by web browsers and served to the user but not all the browsers make the same interpretation of the same code. This is a problem and HTML5 is not an exception. There are some tools to make our life easier like modernizr, which allows you to know which functions are enable in the current user's browser. To see the code of any web page you just need to make right click in every web page and choose an option similar to "view source code" to get a new window with all this information

2. Understanding ASP.NET

This component is a part of our web site that it will be processed within the server before we send the web page to the user. You get more functionality in your web pages by using asp.net allow you to use your web as a normal application with: server-side web framework, provides dynamic content, uses HTML and code-behind, hosts (Internet Information Service IIS) web applications on a server and provides data connectivity (Microsoft SQL Server)

When use asp you can create your own classes in your web project like in a normal console/desktop application. Besides you can add code (VB/C#) directly within your web pages using asp syntax (aspx or razor syntax). This means you will be able to run loops, add variables, call methods and all within your HTML page, which gives great functionality to your future web projects.

3. Hosting Web sites

This is a very important point because when you finish your first web application you will need to host it (save&serve it) in a server that allows you to run asp web pages (if that was the engine you worked with). At this point, IIS takes a really important role due to is going to be en charged of running that application and serve it to the users. During your development you probably faced "IIS Express" running in your windows clock, that is a small version of a current IIS within a server.

4. Intro Web services

A web service is hosted in a web server ;) and typically it does not have any interface to interact with. A example of web service is one created by several weather forecast web pages, which allows developers to access some pieces of code where you can pull useful information like temperature, forecast and weather for a certain town/city. This web services could be interfaces that are implemented by some classes and we, as clients of that web service, we can use that functionality directly. There are a lot of services in the net like Google maps, coordinate, image recognition, even beer location!!!

Cheers!



In this example I am going to show you how to consume with C# a third party image recognition web service (REST). First we need to create a new (30 days free) account in our web service provider: www.recognize.im. Remember to save your account settings to make the connection: client_id, clapi_key and api_key. Then upload some images to your new account and save these images in small size to be used by our new application. I used some star wars images, you will see in the code below. Once all of these steps are finished we can start with our app.


Create a new solution in your VS with two projects inside. A console application will be the interface between the user and the second project, a library which saves all the logic for the web service.
Here below you can see the interface application stored in our console project. It gives some image options to our user. Then the user select one by writing the number reference and we make a call to our library.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Collections;
using OneFlowApp;

namespace OneFlowApp
{
    class Program
    {
        //Data members
        private static string client_id;
        private static string clapi_key;
        private static string api_key;
        private static string url;
        private static Dictionary<int, string> images;

        private static void Main(string[] args)
        {
            Program p = new Program();
            p.run();
        }

        private Program()
        {
            client_id = ""; //API User ID
            clapi_key = ""; //CLAPI user key
            api_key = ""; //API user key
            url = @"http://recognize.im/v2/recognize/single/";
            images = new Dictionary<int, string> {
                    {1, "anakinT.jpg"}, {2, "kenobiT.jpg"}, {3, "lukeT.jpg"},
                    {4, "vaderT.jpg"}, {5, "winduT.jpg"}, {6, "yodaT.jpg"}};
        }

        private void run()
        {
            Console.WriteLine("\n|-| Star Wars characters recognition system |-|\n");
            Console.WriteLine("\rPlease select a character (1-6) from the list below and press enter:\n");
            Console.WriteLine("1.Anakin\n2.Kenobi\n3.Luke\n4.Vader\n5.Windu\n6.Yoda");

            string opt = Console.ReadLine();
            short selection;
            if (Int16.TryParse(opt, out selection)) {
                if (selection > 0 && selection < 7) {
                    string imagePath = AppDomain.CurrentDomain.BaseDirectory + "..\\..\\Resources\\" + images[selection].ToString();
                    RestService rest = new RestService(api_key, url);
                    Dictionary<string, object> response = rest.sendData(imagePath);
                    if (response.ContainsKey("status") && response["status"].ToString() == "0") {
                        ArrayList responseArray = (ArrayList)response["objects"];
                        Dictionary<string, object> responseImage = (Dictionary<string, object>)responseArray[0];
                        Console.WriteLine("Match found! ID: " + responseImage["id"].ToString() + ", Name: " + responseImage["name"].ToString());
                    }
                    else if (response["status"].ToString() == "1") {
                        Console.WriteLine("Error: " + response["message"].ToString());
                    }
                    else if (response["status"].ToString() == "2") Console.WriteLine("No match found.");
                    else Console.WriteLine("No match found.");
                }
                else Console.WriteLine("Not valid option.");
                Console.ReadLine();
            }
            else {
                Console.WriteLine("Error: wrong selection");
                Console.ReadLine();
            }
        }
    }
}
In this part of the code we read the answer from the recognize server to know if it has been able to recognize the image sent. We get different kind of errors and only number “0” means that everything worked fine. In this case we can show the image reference name in the console view.