WHAT'S NEW?
Loading...
Showing posts with label files. Show all posts
Showing posts with label files. Show all posts

Index


  • Introduction
  • Files directories and drives
  • Streams
  • Network
  • Async IO streams
  • References

Introduction

This post is part of a series of post to help you prepare the MCSD certification, particularly the certification exam 70-483, based on the book:


You will find all the code available in the following GitHub repository. Lets talk a little bit about threads and how they work using a .Net development environment.

Files directories and drives


Working with the System.IO namespace you'll find a lot of classes related with files, paths, drives, folders. See the main classes in this list:
  1. File
    1. Exists(string path)
    2. Delete(string path)
    3. Move(string source, string destination)
    4. Copy(path, destPath)
  2. FileInfo
    1. Exists: property
    2. Delete()
    3. MoveTo(string destination)
    4. CopyTo(destPath)
  3. Path static class within the System.IO namespace
    1. Combine(folder, fileName): returns a string with the full path for that file.
    2. GetDirectoryName(path)
    3. GetExtension(path)
    4. GetFileName(path)
    5. GetPathRoot(path)
  4. DriveInfo: it enumerates the current drives and display some information about them
    1. GetDrives()
    2. Name / DriveType / VolumeLabel / DriveFormat
  5. Directory
    1. CreateDirectory()
    2. GetFiles(string path)
  6. DirectoryInfo (multiple operations against a folder)
    1. Create()
    2. GetDirectories(): searchPattern can be passed as a parameter to reduce the amount of directories retrieved.
    3. EnumerateDirectories(): with GetDirectories you get a list and you need to wait until is completely retrieved but with this you can start enumerating the directories before been completly retrieved.
    4. MoveTo()
    5. Move()
    6. GetFiles();
  7. SearchOptoin: specify a search pattern to be performed agains a Directory/DirectoryInfo object
    1. wildcards: * stands for any group of characters, ? stands for any single character.
  8. DirectorySecurity: grant access to folders, hosted in System.Security.AccessControl namespace.

Insufficient privileges creating a folder can end up throwing a UnathorizedAccessException and trying to remove a folder that doesn't exists in a DirectoryNotFoundException.

Streams


Streams refers to abstractions of a sequence of bytes, which means, if you want to save text first you need to transform that text into an array of bytes. Types of streams:
  • Reading
  • Writing: when writing to a FileStream
    • Write(byte array)
  • Seeking: stands for some streams have the concept of a current position.
To convert between bytes and strings we use the System.Text.Enconding class, it uses different standards to make the transformation like:
  • UTF-8
  • ASCII
  • BigEndianUnicode
  • Unicode
  • UTF32
  • UTF7
To follow this process in an easier way, the File class supports a CreateText() method which returns a StreamWriter object that we can use to perform write operations directly to a file. To read the data you can use a FileStream object, you would need to read every byte one by one by using the ReadByte() method and finally, translate those bytes using a particular encoding like UTF8 calling the method: Encoding.UTF8.GetString(data) which returns a string. The alternative, if you know you are reading a piece of text, is the StreamReader and call the method ReadLine(). See example below:

using System.IO; using System.Text; namespace Chapter4 { public static class CompareFileStreamReader { public static void HowToReadAFile() { string path = @"c:\temp\test.txt"; // read a file using a FileStream object using (FileStream fileStream = File.OpenRead(path)) { byte[] data = new byte[fileStream.Length]; for (int index = 0; index < fileStream.Length; index++) { data[index] = (byte)fileStream.ReadByte(); } System.Console.WriteLine(Encoding.UTF8.GetString(data)); } // read a file using a StreamReader object using(StreamReader reader = File.OpenText(path)) { System.Console.WriteLine(reader.ReadLine()); } } } }

Working with multiple streams together (decorator pattern) can be handy sometimes, specially when compressing/decompressing streams (FileStream, MemoryStream,...) using the GZipStream class within the System.IO.Compression namespace.

Another interesting class is BufferedStream, which comes handy when you need to read/write big chunks of data (which is what drives are optimized for, instead of reading byte by byte). It wraps a FileStream object and works out if it's possible to read larger chunks of data at once.

To finish this chapter about streams, I want to highlight how files can be locked by the OS and you might think the file was deleted. For example a File.Exists call can return false and we assume there's no such a file, while in reality, the file is in use by another thread. This is important to consider when writing/reading resources and always wrap our code up with try/catch blocks. Typically we want to focus on DirectoryNotFoundException and FileNotFoundException.

Network


The .Net framework enable your apps to communicate over the network. It's then when the System.Net namespace turns interesting and the WebRequest and WebResponse classes vital. There are specific implementations by protocol like HttpWebRequest and HttpWebResponse based on the protocol HTTP. See an usage example here:


Async IO streams


All the classes showed in this post until now are synchronous classes. Working with resources, typically, implies some delays we need to take in consideration. If not we can make our apps looks unresponsive (it happens when an app looks freezes and the user can't perform any action). 

C#5 introduced async/await keywords, this will make the compiler responsible for turning your "synchronous-looking" code into a state machine that handles all possible situations. Many methods have an async equivalent that returns Task or Task<T>, if there's nothing in return or if the method returns a type (T), respectively. The use of this async methods will make your apps more responsible and scalable

Real async makes sure your thread can do other work until the OS notifies the I/O is ready. Bear in mind not to use the await operator multiple times within the same function if the async calls can be run in parallel. The following example shows you how you need to call the await operator at the end of your functions to just wait the minimum possible time, in this case, wait only for the longest call, instead of for each individual.


Index

  • Intro
  • File System
  • Modify files
  • What are web services
  • REST Services
  • Asynchronous programming

Intro

I want to talk today about how to work with the file system and why we need it, so let's talk about the different scenarios where you'll be using the System.IO .Net library.


Besides we'll be scratching the surface of a couple of web services called "SOAP" and "REST", which you could be already familiar with. Finally, asynchronous programming will come over.

File System

Benefits of read and write from the file system:

  • Show existing data to user
  • Integrate user-provided data
  • Serialize objects out of memory
  • Persist data across sessions
  • Determine environment configuration

How to write:

  • This is simplified with Framework methods; 
    • open / shut (File.WriteAllText / File.ReadAllText) (see example below)
  • Open for reading to keep open and keep writing.
  • Open as stream for large payloads and real-time processing: let's say you want to see a movie through Netflix, you don't download the entire movie file into your PC and then you start watching. You download just a small peace of that huge file depending on where you are in the movie.
See an example of how write/read a file using .Net


var dir = System.IO.Directory.GetCurrentDirectory();
var file = System.IO.Path.Combine(dir, "File.txt");
var content = "how now brown cow?";

// write
System.IO.File.WriteAllText(file, content);

// read
var read = System.IO.File.ReadAllText(file);
Trace.Assert(read.Equals(content));


How do we find files?

  • Get Windows folder with Environment Special Folder
  • Get the Current folder with File.IO.GetCurrentDirectory()
  • Use Isolated Storage dedicated to the current application: 
    • dedicated application storage not used very often by developers.
  • Anything Else.Caveat: Windows Store App development

// special folders
var docs = Environment.SpecialFolder.MyDocuments;
var app = Environment.SpecialFolder.CommonApplicationData;
var prog = Environment.SpecialFolder.ProgramFiles;
var desk = Environment.SpecialFolder.Desktop;

// application folder
var dir = System.IO.Directory.GetCurrentDirectory();

// isolated storage folder(s)
var iso = IsolatedStorageFile
    .GetStore(IsolatedStorageScope.Assembly, "Demo")
    .GetDirectoryNames("*");

// manual path
var temp = new System.IO.DirectoryInfo("c:\temp");

In the previous example you can see how to reach the typical windows directories, your app and your isolated storage folder in .Net. Finally directoryInfo provides you useful info about certain directory like size, rights, etc. You are not allowed to do whatever you want in your PC, at some folder you probably need some rights in order to read, write or... delete!

Modify files

Please see here a small piece of code to show you how to edit files, following these steps:

  • Iterate through files using GetFiles()
  • Rename / Move with System.IO methods
  • Get File Info with System.UI.FileInfo

// files
foreach (var item in System.IO.Directory.GetFiles(dir))
    Console.WriteLine(System.IO.Path.GetFileName(item));

// rename / move
var path1 = "c:\temp\file1.txt";
var path2 = "c:\temp\file2.txt";
System.IO.File.Move(path1, path2);

// file info
var info = new System.IO.FileInfo(path1);
Console.WriteLine("{0}kb", info.Length / 1000);


See how we get all the files defined in the "dir" variable (let say "C:\temp") using GetFiles. Then we print all those files placed in the "dir" directory using GetFileName. Then moving a file to the same folder which purely it's a renaming operation. Finally, we'll get some meta data using the FileInfo method and we use the result of that method to print on the console the length of that file.

What are web services

See some web services (WS) features:

  • Web services encapsulate implementation: this means we don't know how a third party service is implemented. You can be using Facebook web services but you don't know how their WS are implemented.
  • WS expose to disparate system.
  • WS allow client systems to communicate servers: which is great because allow you to separate the different parts when you're building a new application.
    • Web protocols (HTTP, GET, POST, etc)
  • WS are important to Service Oriented Architecture
    • With and without metadata
    • Loose coupling: when a system has different moving parts depending on others and add something new means add a lot of extra changes.

What is SOAP? Easy, it's a type of web service. See some features:


  • SOAP is a standard for returning structured data from a Web Service as XML
    • Envelope (Header / Body / ...)
  • SOAP handling is a built-in feature of Visual Studio
The idea is to serialize the information provided by our application into our web service to send that information to our clients. Then our clients will de serialize that information in order to be able to read it.

The downside of this is the de/serialization process is heavy. At first, save 400 bytes it's not a big deal, but in case of companies like Facebook which could face millions of requests per day means a lot. There is a few emerging new technologies to bypass this problem...

REST Services

REST (Representative State Transfer)

  • ...is becoming a common, industry standard
  • ...does not require XML parsing
  • ...does not require message header
  • ...is generally human-readable
  • ...uses less band-with than SOAP
  • ...services typically return XML or JSON
  • ...JSON is JavaScript Object notation
    • JSON is becoming a common industry standard
    • JSON is generally a lighter payload than XML (or SOAP)

See how to implement a basic JSON reader using the .Net framework:

using System;
using System.Collections.Generic;
using System.Net;
using System.Web.Script.Serialization;

namespace ConsoleApplication1
{
    public class Program
    {
        public static void Main()
        {
            var url = new Uri("http://footballpool.dataaccess.eu/data/info.wso/TopGoalScorers/JSON/debug?iTopN=10");
            var client = new WebClient();
            var json = client.DownloadString(url);

            // deserialize JSON into objects
            var serializer = new JavaScriptSerializer();
            var data = serializer.Deserialize>(json);

            // use the objects
            foreach (var item in data)
                Console.WriteLine("Player: {0,16}, Goals: {1}", item.sName, item.iGoals);

            Console.Read();

            // Output
            // Player:  James Rodríguez, Goals: 6
            // Player:   Thomas Mueller, Goals: 5
            // Player:            Messi, Goals: 4
            // Player:           Neymar, Goals: 4
            // Player:       van Persie, Goals: 4
            // Player:   Andre Schurrle, Goals: 3
            // Player:          Benzema, Goals: 3
            // Player:   Enner Valencia, Goals: 3
            // Player:    Islam Slimani, Goals: 3
            // Player:           Robben, Goals: 3
        }
    }

    public class ObjectReturnedByTheWebService
    {
        public string sName { get; set; }
        public int iGoals { get; set; }

    }
}


If you want to try this solution, you could create a new console application and then add a new reference to System.Web.Extensions in order to get the JavaScriptSerializer class working. The URL I'm using it's a free web service with free football stats called: http://footballpool.dataaccess.eu/. I'm passing the value 10 when building the URL, but you can change that in order to get top3, top5...

I use the WebClient class to call the service and then I de-serialize the message because it comes as a JSON message. Try to go to that URL to see how it looks. There is a small class called ObjectReturnedByTheWebService which is used to convert that json message from plain text into a proper .Net class using a auto mapper.


As an example of how using REST services, please see how to implement an image recognition system in one of my first posts for this blog.

Asynchronous programming

Asynchronous maximizes resources on multi core systems, by allowing units of work to be separated and completed. Asynchronous programming frees up the calling system, especially a user interface, as to not wait for long operations. Don't using this approach will constrain your resources. It improves the user experience using your application by freezing the screen giving a feel like your app is hanging in a long running task.

References