WHAT'S NEW?
Loading...
Showing posts with label async. Show all posts
Showing posts with label async. Show all posts

Index


  • Introduction
  • Files directories and drives
  • Streams
  • Network
  • Async IO streams
  • References

Introduction

This post is part of a series of post to help you prepare the MCSD certification, particularly the certification exam 70-483, based on the book:


You will find all the code available in the following GitHub repository. Lets talk a little bit about threads and how they work using a .Net development environment.

Files directories and drives


Working with the System.IO namespace you'll find a lot of classes related with files, paths, drives, folders. See the main classes in this list:
  1. File
    1. Exists(string path)
    2. Delete(string path)
    3. Move(string source, string destination)
    4. Copy(path, destPath)
  2. FileInfo
    1. Exists: property
    2. Delete()
    3. MoveTo(string destination)
    4. CopyTo(destPath)
  3. Path static class within the System.IO namespace
    1. Combine(folder, fileName): returns a string with the full path for that file.
    2. GetDirectoryName(path)
    3. GetExtension(path)
    4. GetFileName(path)
    5. GetPathRoot(path)
  4. DriveInfo: it enumerates the current drives and display some information about them
    1. GetDrives()
    2. Name / DriveType / VolumeLabel / DriveFormat
  5. Directory
    1. CreateDirectory()
    2. GetFiles(string path)
  6. DirectoryInfo (multiple operations against a folder)
    1. Create()
    2. GetDirectories(): searchPattern can be passed as a parameter to reduce the amount of directories retrieved.
    3. EnumerateDirectories(): with GetDirectories you get a list and you need to wait until is completely retrieved but with this you can start enumerating the directories before been completly retrieved.
    4. MoveTo()
    5. Move()
    6. GetFiles();
  7. SearchOptoin: specify a search pattern to be performed agains a Directory/DirectoryInfo object
    1. wildcards: * stands for any group of characters, ? stands for any single character.
  8. DirectorySecurity: grant access to folders, hosted in System.Security.AccessControl namespace.

Insufficient privileges creating a folder can end up throwing a UnathorizedAccessException and trying to remove a folder that doesn't exists in a DirectoryNotFoundException.

Streams


Streams refers to abstractions of a sequence of bytes, which means, if you want to save text first you need to transform that text into an array of bytes. Types of streams:
  • Reading
  • Writing: when writing to a FileStream
    • Write(byte array)
  • Seeking: stands for some streams have the concept of a current position.
To convert between bytes and strings we use the System.Text.Enconding class, it uses different standards to make the transformation like:
  • UTF-8
  • ASCII
  • BigEndianUnicode
  • Unicode
  • UTF32
  • UTF7
To follow this process in an easier way, the File class supports a CreateText() method which returns a StreamWriter object that we can use to perform write operations directly to a file. To read the data you can use a FileStream object, you would need to read every byte one by one by using the ReadByte() method and finally, translate those bytes using a particular encoding like UTF8 calling the method: Encoding.UTF8.GetString(data) which returns a string. The alternative, if you know you are reading a piece of text, is the StreamReader and call the method ReadLine(). See example below:

using System.IO; using System.Text; namespace Chapter4 { public static class CompareFileStreamReader { public static void HowToReadAFile() { string path = @"c:\temp\test.txt"; // read a file using a FileStream object using (FileStream fileStream = File.OpenRead(path)) { byte[] data = new byte[fileStream.Length]; for (int index = 0; index < fileStream.Length; index++) { data[index] = (byte)fileStream.ReadByte(); } System.Console.WriteLine(Encoding.UTF8.GetString(data)); } // read a file using a StreamReader object using(StreamReader reader = File.OpenText(path)) { System.Console.WriteLine(reader.ReadLine()); } } } }

Working with multiple streams together (decorator pattern) can be handy sometimes, specially when compressing/decompressing streams (FileStream, MemoryStream,...) using the GZipStream class within the System.IO.Compression namespace.

Another interesting class is BufferedStream, which comes handy when you need to read/write big chunks of data (which is what drives are optimized for, instead of reading byte by byte). It wraps a FileStream object and works out if it's possible to read larger chunks of data at once.

To finish this chapter about streams, I want to highlight how files can be locked by the OS and you might think the file was deleted. For example a File.Exists call can return false and we assume there's no such a file, while in reality, the file is in use by another thread. This is important to consider when writing/reading resources and always wrap our code up with try/catch blocks. Typically we want to focus on DirectoryNotFoundException and FileNotFoundException.

Network


The .Net framework enable your apps to communicate over the network. It's then when the System.Net namespace turns interesting and the WebRequest and WebResponse classes vital. There are specific implementations by protocol like HttpWebRequest and HttpWebResponse based on the protocol HTTP. See an usage example here:


Async IO streams


All the classes showed in this post until now are synchronous classes. Working with resources, typically, implies some delays we need to take in consideration. If not we can make our apps looks unresponsive (it happens when an app looks freezes and the user can't perform any action). 

C#5 introduced async/await keywords, this will make the compiler responsible for turning your "synchronous-looking" code into a state machine that handles all possible situations. Many methods have an async equivalent that returns Task or Task<T>, if there's nothing in return or if the method returns a type (T), respectively. The use of this async methods will make your apps more responsible and scalable

Real async makes sure your thread can do other work until the OS notifies the I/O is ready. Bear in mind not to use the await operator multiple times within the same function if the async calls can be run in parallel. The following example shows you how you need to call the await operator at the end of your functions to just wait the minimum possible time, in this case, wait only for the longest call, instead of for each individual.


Index


  • Introduction
  • Using the Parallel class
  • Using async and await
  • Using the Parallel Language Integrated Query (PLINQ)
  • Using concurrent collections

Introduction

This post is part of a series of post to help you prepare the MCSD certification, particularly the certification exam 70-483, based on the book:


You will find all the code available in the following GitHub repository. Lets talk a little bit about threads and how they work using a .Net development environment.

Using the Parallel class

System.Threading.Tasks contains also another class we have not checked yet in previous posts. The Parallel class has a couple of static methods (For, ForEach and Invoke). This doesn't mean you should replace all your loops with parallel loops.

Use the parallel class only when your code doesn't have to be executed sequentially. When you have a lot of work to be done that can be executed in parallel. For smaller work sets or for work that has to synchronize access to resources, using the Parallel class can hurt performance.

In the following example you can see how to use the method For and ForEach when you want to run parallel operations without sharing resources. The third block of code is using the ParallelLoopResult to show you how to break a running operation when the loop completes the first half of iterations. At that point, the IsCompleted value equals to false and the LowestBreakIteration is 500.


Using async and await

When accessing I/O components on the primary thread, Windows typically, pauses your thread in order to release the CPU. Asynchronous solves this limitation but it has some complexity when writing code. For this reason, Microsoft introduced async and await operators when C#5 was released improving responsiveness and scalability.

A method marked with async starts on the current thread and it contains calls marked with await, which are our parallel tasks. Using the await keyword, the compiler can be aware if an operation has finished (continuing synchronously) or not (hooking up a continuation method that should run when the Task completes.

In the following example you get the taste of how this great tool works. Note: you probably need to import the System.Net.Http dll into your project if your IDE can't resolve the HttpClient class dependency



See how a separated method is decorated with the "async" keyword because the entry point for an app can't be marked as async.

Keep in mind, you are not making your app performing any better just by using tasks and awaiting them. You're improving responsiveness.

Normally, when an exception happens you expect an AggregateException to be triggered.

Another relevant point is the SynchronizationContext which connects its application model to its threading model. It abstracts how different applications work and helps you end up on the right thread working with a WPF app or ASP... This means, for example in an ASP app, tasks will find the right user related information (current user, culture,...) when get finished.

In some cases you don't need this data and could run an asynchronous task without the context. Calling the method ConfigureAwait(false) you disable the context saving process and you get better performance. See how in the following example we use a couple of awaits without saving the context. We are getting some text from internet and placing those lines in a text file.



With async/await you should keep in mind:

  1. Never return void within an async method. Return void is only valid for asynchronous events.
  2. Never marked a method as async without any await call decorated.

Using the Parallel Language Integrated Query (PLINQ)

As you are probably aware, LINQ (Language –Integrated Query) is a popular syntax used to perform queries over all kind of data. PLINQ is the parallel version and lets you run extended methods like for example: Where, Selet, SelectMany, GroupBy, Join, OrderBy, Skip and Take.

It's the runtime who decided weather to run in parallel or not. You can force to run in parallel by calling: WithExecutionMode() method. Behind the scenes the CLR will generate different Tasks objects to perform the asynchronous process.

By default, PLINQ uses all the available processor in the CPU (up to 64) but you can always limit this by calling: WithDegreeOfParallelism() method, and passing an integer which represents the number of processors to be used. Note, this parallelism does not guarantee any particular order, see following example:


There is also a way to get this results ordered by calling AsOrdered() function, which still perform a parallel action but buffering the results from the different tasks.

In some cases you might want to perform a sequential operation after a parallel one. By calling AsSequential(), PLINQ will return the answer as non parallel. In the following example we are using Take() and it might mess up the result unless we call AsSequential() first:


On the other hand, if you don't mind get not ordered results you might want to perform a ForAll() function call: parallelResult.ForAll(e => Console.WriteLine(e)); It does not need all the results before starting, but, it will remove any sort order specified. In some cases, you can end up with exceptions in your queries which will be handled using AggregateException. This will show a list of all exceptions triggered during the parallel execution.

Using concurrent collections

Asynchronous access to data always need to be concurrent safe, it means, more than one thread can modify the same data at the same time. There are some collections in .Net developers use to ensure concurrency is safe used like:

  • BlockingCollection<T>
Thread-safe entity handles access to its information. In reality it's a wrapper around a collection, by default, the ConcurrentQueue. For this example, notice how there is one task which listens for new items being added to the collection blocking if no items are available, while other task adds items to it.


You could call the CompleteAdding() method to signal that no more items will be added so other threads waiting for items won't be blocked any more. We can improve the algorithm by using GetConsumingEnumerable() method which returns an Enumerable object that blocks until it finds a new item. With this you can use a foreach with your BlockingCollection to enumerate it.

  • ConcurrentBag<T>
It is, effectively, just a bag which allow you to perform Add, TryTake and TryPeek with duplicates an no particular order. Sometimes I found TryPeek not very useful because you can end up with another tread deleting the item before you actually access it. ConcurrentBag implements IEnumerable<T> so you can iterate over an snapshot of the collection. If new items are added after you start iterate they won't be visible until you start a new iteration.
  • ConcurrentDictionary<TKey, T>
It stores key and value in a thread-safe manner, you can add, remove and update (if exist) items as atomic operations (a single operation will be performed without other threads interfering). TryUpdate() will check first if the current value is different from the new one. AddOrUpdate() will push a new item to the Dictionary if it doesn't exists. GetOrAdd() gets the current value of an item if it's available, if not, it will add the new value.
  • ConcurrentQueue<T>
  • ConcurrentStack<T>
Stack is a LIFO collection and a Queue is a FIFO collection. ConcurrentStack implements Push() (save item) and TryPop() (get item) methods, unfortunately you can never be sure whether are items within the queue due to several threads running in parallel. You can also add or remove bulk items with: PushRange() and TryPopRange().

ConcurrentQueue provides Enqueue (push) and TryDequeue (get) to add and remove items from the queue. It also contains a TryPeek() and it implements the IEnumerable class by making a snapshot of the data.


References

https://www.microsoftpressstore.com/store/exam-ref-70-483-programming-in-c-sharp-9780735676824
https://github.com/tribet84/MCSD/tree/master/mcsd

Index

  • Intro
  • Async and Await
  • What is a database
  • What is LINQ
  • LINQ Demo

Intro

Let's talk today about data. Bring data is typically a high cost operation that's why we need to use always the perfect data storage system regarding our requirements. There are techniques like asynchronous communications which will help us on speeding our systems. How to retrieve that data is an important subject and decide which approach to use will be also discussed.


Async and Await

Async and await simplify asynchronous programming in .Net. Async and await allow asynchronous code to resemble the structure of synchronous code. Methods marked with async may return Task as the Task-based Asynchronous Pattern (TAP) defines. The async keyword instructs the compiler to allow await. The await keyword instructs the methods to return The await keyword instructs the compiler to resume execution within the same context after the operation is complete. 

In the following example we run an asynchronous operation in our call to DownloadStringAsync which is defined within the WebClient component in the .Net framework. This functions performs some operation in background, which means, the execution process in our app is never stopped. We don't know when this task will be completed, that's why we've published an event called "Completed" which a class can subscribed to and be notified when the completion occurs.



using System;
using System.Net;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class AsynchronousExample
    {
        public event EventHandler Completed;

        void GetHtml(string url)
        {
            var client = new WebClient();
            client.DownloadStringCompleted += client_DownloadStringCompleted;
            client.DownloadStringAsync(new Uri(url));
        }

        private void client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            if (Completed != null)
                Completed(this, e);
        }
    }
}


In our next example we are using the reserved word "async" to decorate our method. See the type returned here is a Task. This means when we trigger probably a long running process which will return immediately a Task object. It won't stop to the completion of the asynchronous method. This task object will contain the result but it will be uncompleted until the DownloadStringTaskAsync finishes.


using System;
using System.Net;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class AsynchronousExample
    {
        private async Task GetHtmlWithAsync(string url)
        {
            var client = new WebClient { };
            return await client.DownloadStringTaskAsync(new Uri(url));
        }
    }
}

What is a database

  • A database is a data store
  • Data stored in a database is structured
  • Data stored in a database may be related (Referential integrity)
  • A database provides a way to access / query data (Optimized by indexes)

SQL DB Types

  • Windows Azure SQL Database
  • Local Network SQL Database
  • Local Machine SQL Server Express 
  • Application SQL LocalDB
  • Application SQL CE
  • Other providers: Oracle, SqLite, MySql, DB2

Types of access to a database: how do we talk to the database
  • Low-Level
    • Manual queries
    • DbDataReader
  • Object Relationship Models (ORM): allow you to conceptualize the model of you db.
    • Entity Framework
    • nHibernate

What is LINQ

Language Integrated Query:

  • is a general-purpose Query Language
  • is an integrated part of the .Net languages
  • is Type Safe and has Intellisense
  • includes operators like Traversal, Filter and Projection
  • can be optimized with compiler versions
  • can be invoked using its Query Syntax
  • can be invoked using its Method Syntax

LINQ Demo

See here enclosed some examples about how to use the LINQ library in a .Net environment. I'll try to show you how to get the even and odds number from a list. Then we'll move forward sorting a bunch of letters in a handy way

var data = Enumerable.Range(1, 50);

// even or odd
// method syntax
var method = // IEnumerable
    data.Where(x => x % 2 == 0)
        .Select(x => x.ToString()); // Projection: take data and transform it into text

// query syntax
var query = // IEnumerable
    from d in data
    where d % 2 == 0
    select d.ToString();

var projection =
    from d in data
    select new
    {
        Even = (d % 2 == 0),
        Odd = (d % 2 != 0),
        Value = d,
    };


I'm using different approaches in order to get odd's and even numbers. I'm presenting the result in different ways also. For the method and the query variable, I'm storing the values as strings. Within the projection variable, I'm actually creating small structures containing three values, the first two save a boolean true/false and the last one stores the current value.

var letters = new[] { "A", "C", "B", "E", "Q" };

// query syntax
var sortAsc =
    from d in data
    orderby d ascending
    select d;

// method syntax
var sortDesc =
    letters.OrderByDescending(x => x);


var values = new[] { "A", "C", "A", "C", "A", "D" };

var distinct = values.Distinct();   // remove duplicates
var first = values.First();         // error if "values" is empty
var firstOr = values.FirstOrDefault();  // default if "values" is empty
var last = values.Last();
var page = values.Skip(2).Take(2);


See how in the previous example I'm working with a couple of character lists. For the first list I sort the characters using linq query sintax and linq method syntax respectively. For the last list, I wanted to show you the potential of  this language which allows you to performs actions like, remove duplicates, get the first/last or even skip some values and take more than one.

Really powerful library which I'm sure will help you to short out your code and present your future projects in a really professional way.

References

http://www.microsoftvirtualacademy.com/training-courses/developer-training-with-programming-in-c
https://msdn.microsoft.com/en-us/library/hh873175.aspx
https://msdn.microsoft.com/es-es/library/bb397926.aspx