.NET Stream and File Operations

This chapter is from the book

Visual Basic Programmer's Guide to the .NET Framework Class Library

Learn More Buy

This chapter is from the book

This chapter is from the book 

Visual Basic Programmer's Guide to the .NET Framework Class Library

Learn More Buy

Reading and Writing to Files and Streams

As programmers, we often have to write directly to a file or data stream. If you've communicated with disparate systems, you are undoubtedly familiar with writing out CSV or XML files as a means of exchanging data. The .NET Framework gives us a group of classes and methods inside the System.IO namespace that allows us to access data streams and files both synchronously and asynchronously.

This section will define the AppendText, CreateText, Open, OpenRead, and OpenWrite methods of the FileInfo class.

File and data streams are essentially the same thing. They both are a type of stream. Their differences lie in their backing store. Backing store refers to a storage medium, such as a disk, tape, memory, network, and so on. Every backing store implements its own stream type as a version of the Stream class. This allows each stream type to read and write bytes to and from its own backing store. These streams that connect directly to backing stores are called base streams in .NET. An example of a base stream is the FileStream class. This class gives us access to files stored on disk inside of directories.

Reading and Writing Files

To read data to and from files using the System.IO namespace, we primarily use two classes: FileInfo and FileStream. The FileInfo class exposes a number of methods that allow us access to stream-related functions based on a file. These methods simply use the FileStream and related classes to expose this functionality. However, they are useful as you often already have an instance of FileInfo that is specific to a given file. You can then call these methods to return and write to the contents of this file. Table 7.11 lists these methods and their associated return types.

TABLE 7.11 FileInfo Streaming Methods

You can see that the FileInfo class makes extensive use of the FileStream, StreamWriter, and StreamReader classes. These classes expose the necessary functionality to read and write to files in .NET. As you might have guessed, these objects are designed to work with persisted text files. They are based on the TextWriter and TextReader classes.

The FileStream class can be created explicitly. You've already seen that FileInfo uses this class to expose reading and writing to files. Table 7.12 lists the version of the FileStream constructors that can be used to create a FileStream object.

TABLE 7.12 FileStream Constructors

Listing 7.5 provides an example of the FileStream, StreamWriter, and StreamReader classes. This example is a simple, console-based application. It creates a new FileStream object based on a physical file. It then creates a StreamWriter instance based on the FileStream class. It calls the WritLine method of StreamWriter to output a line of text to the file. After it closes the StreamWriter instance, it creates a StreamReader instance based on a FileStream object. Finally, it loops through the lines in the file and outputs them to the console for your viewing.

LISTING 7.5 FileStream, StreamWriter, and StreamReader

Imports System.IO 

Module Module1 

   Sub Main() 

     'purpose: open a file and append infromation to its end 

     'local scope 
     Dim fileStream As FileStream 
     Dim streamWriter As StreamWriter 
     Dim streamReader As StreamReader 

     'create a new instance of the file stream object 
     'note:  if the file does not exist, the constructor create it 
     fileStream = New     fileStream(path:="c:\test.txt", _ 
        mode:=FileMode.OpenOrCreate, access:=FileAccess.Write) 

     'create an instance of a character writer 
     streamWriter = New StreamWriter(stream:=fileStream) 

     'set the file pointer to the end of the file 
     streamWriter.BaseStream.Seek(offset:=0, origin:=SeekOrigin.End) 

     'write a line of text to the end of the file 
     streamWriter.WriteLine(value:="This is a test") 
 
     'apply the update to the file 
     streamWriter.Flush() 

     'close the stream writer 
     streamWriter.Close() 

     'close the file stream object 
     fileStream.Close() 

     'create a new instance of file stream to read the file back 
     fileStream = New fileStream(path:="c:\test.txt", _ 
        mode:=FileMode.OpenOrCreate, access:=FileAccess.Read) 

     'create a stream reader instance 
     streamReader = New StreamReader(stream:=fileStream) 

     'set the file pointer to the start of the file
     streamReader.BaseStream.Seek(offset:=0, _ 
        origin:=SeekOrigin.Begin) 

     'loop through the file and write to console until the 
     ' end of file reached 
     Do While streamReader.Peek > -1 
          Console.WriteLine(value:=streamReader.ReadLine()) 
      Loop 

      'close the stream reader 
      streamReader.Close() 

      'wait for the user to stop the console application 
      Console.WriteLine("Press 's' to stop the application.") 

      'loop until users presses s key 
      Do While Console.ReadLine <> "s" : Loop 
   End Sub 
End Module 

Asynchronous Reading and Writing

As stated earlier, streaming with the .NET Framework classes can be done both synchronously and asynchronously. Synchronous reading and writing blocks methods from continuing until the operation is complete. For instance, suppose your application takes orders in the form of text files written to a queue. When a file is placed in the queue (or directory), your application reads the contents of the file and processes the order(s) accordingly. Each file can represent one order, or can contain a batch of orders. If your application is set up to handle each order from start to finish as it comes in (synchronously), then a long order will block your application from continuing to process orders while simply reading the file.

For a more efficient use of your resources, you will want to read orders asynchronously. That is, as an order comes in, you will tell a version of the Stream object to start reading the file and to let you know when it is done. This way, once you fire the BeginRead method, you can continue executing other program logic including responding to and processing additional orders.

With asynchronous file I/O, the main thread of your application continues to execute code while the I/O process finishes. In fact, multiple asynchronous IO requests can process simultaneously. Generally, an asynchronous design offers your application better performance. The tradeoff to this performance is that a greater coding effort is required.

The FileStream class provides us the BeginRead method for asynchronous file input and the BeginWrite method for asynchronous file output. As a parameter to each, we pass the name of the method we wish to have called when the operation is complete (userCallback as AsynchCallback). In VB .NET, the syntax looks like this:

New AsyncCallback(AddressOf myCallbackMethod) 

Where myCallbackMethod is the name of the method you wish to have intercept and process the completed operation notification. From within this callback method, you should call EndRead or EndWrite as the case dictates. These methods end their respective operations. EndRead returns the number of bytes that were read during the operation. Both methods take a reference to the pending asynchronous I/O operation (AsynchResult as IAsynchResult). This object comes to us as a parameter to our custom callback method. The code in Listing 7.6 further illustrates these concepts.

The application's Sub Main simply controls the calls to the read operation. You can see in Listing 7.6 that we execute three separate read requests on three different files. The remaining bits of functionality are nicely encapsulated and thus, should be easy to reuse.

LISTING 7.6 Asynchronous File Reading

Imports System.IO 

Module Module1 

   Sub Main() 
     'purpose: provide example code of asynch. file I/O 
     'steps:    1. start asynch. read and processing of 3 files of 
     '           varying lengths 
     '          2. wait for callback and display to screen 

     'local scope 
     Dim myFiles(3) As String 
     Dim i As Int16 

     myFiles(0) = "c:\file1.txt" 
     myFiles(1) = "c:\file2.txt" 
     myFiles(2) = "c:\file3.txt" 

     'call each asynch. read 
     For i = 0 To 2 
        Console.WriteLine("Starting file read: " & myFiles(i)) 
        Call asynchRead(filePath:=myFiles(i)) 
     Next 

     'NOTE: now that file reads have started, our application can 
     '       continue processing other information and await a callback 
     '       from the read operation indicating read is complete 

     'wait for the user to stop the console application 
     Console.WriteLine("******** Enter 's' to stop the application.") 

     'loop until user presses s key 
     Do While Console.ReadLine <> "s" : Loop 

End Sub 

The procedure asynchRead sets up the asynchronous file input. The class StateObject is a simple state object that allows us to maintain file input information, in the form of properties, across read requests.

Notice that when calling BeginRead, in addition to indicating a callback method, we must specify both a byte array (array() as Byte) and the total number of bytes (numBytes as Integer) we wish to have read. To store the bytes, we dimension an array of type byte inside our state object. We pass byteArraySize in the object's constructor. We get its size by reading the file size from the FileInfo object's Length property. This allows us to create an array of the exact size we need. Similarly, when we set the number of bytes to read, we use FileInfo.Length again to indicate we want to read the entire file.

   Private Sub asynchRead(ByVal filePath As String) 

      'purpose: execute an asynch. read against a given file 
      '        throw an exception if the file is not found 

      'local scope 
      Dim fileStream As FileStream 
      Dim state As StateObject 
      Dim fileInfo As FileInfo 

      'check to see if the file exists 
      If Not File.Exists(path:=filePath) Then 

         'file does not exist = throw an exception 
         Throw New Exception(message:="File not found.") 

      End If 

      'file exists = create an open instance of the file 
      fileStream = New FileStream(path:=filePath, mode:=FileMode.Open) 

      'determine size of the file to set the number of bytes 
      fileInfo = New FileInfo(fileName:=filePath) 
      Console.WriteLine("File length: " & fileInfo.Length) 

       'create a state object 
       state = New StateObject(filePath:=filePath, _ 
          byteArraySize:=fileInfo.Length) 

       'set fileStream prop (useful for callback) 
       state.FileStream = fileStream 

       'begin the file read 
       fileStream.BeginRead(array:=state.ByteArray, offset:=0, _ 
         numBytes:=fileInfo.Length, _ 
         userCallback:=New AsyncCallback(AddressOf fileRead), _ 
         stateObject:=state) 

   End Sub 

The fileRead method is the application's callback implementation. This method receives notification when a BeginRead has completed for a given file.

   Private Sub fileRead(ByVal asyncResult As IAsyncResult) 

     'purpose: provide a callback method for asynch reads 
 
     'local scope 
     Dim state As StateObject 
     Dim bytesRead As Integer 

     'set the state object = to the one returned by the asynch results 
     state = asyncResult.AsyncState 

     'write out the path of the object read 
     Console.WriteLine(state.FilePath) 

     'indicate that the file was read asynch. 
     If asyncResult.CompletedSynchronously Then 
        Console.WriteLine("File was read synchronously.") 
     Else 
        Console.WriteLine("File was read asynchronously.") 
     End If 

     'determine the number of bytes read by calling EndRead 
     bytesRead = state.FileStream.EndRead(asyncResult) 

     'write out bytes read 
     Console.WriteLine("Bytes read: " & bytesRead) 

     'close the file stream 
     state.FileStream.Close() 

   End Sub 

End Module 

Public Class StateObject 

   'purpose: maintain state information across asynch calls 

   'class-level scope 
   Private localFilePath As String 
   Private localByteArray() As Byte 

   'public properties 
   Public FileStream As FileStream 

   Public Property ByteArray() As Byte() 
     'purpose: get and set ByteArray property 

     Get 
        Return localByteArray 
     End Get 

     Set(ByVal Value() As Byte) 
        localByteArray = Value 
     End Set
 
     End Property
 
   Public Property FilePath() As String 

     'purpose: get and set FilePath prop. 

     Get 
        Return localFilePath 
     End Get 

     Set(ByVal Value As String) 
        localFilePath = Value 
     End Set 

   End Property 

   Sub New(ByVal filePath As String, ByVal byteArraySize As Integer) 

      'purpose:  constructor, allows setting of file path info. 
      '         and byte array size info. 

      'set local file path info 
      localFilePath = filePath 

      'dimension the size of the byte array 
      ReDim localByteArray(byteArraySize) 

  End Sub 
End Class 

Figure 7.2 represents the output of the code listing. Notice that in this case, each file was read in the same order the request was made. However, there is no guarantee of processing order due to the asynchronous nature of the request and additional factors like file size and processor availability. Also notice that after the first (and subsequent) read requests were made, our code did not stop executing. Rather, it made additional requests, and ultimately, waited on user input to stop the application. Finally, as each read completed, the notification was sent to our readFile method and the results of the operation were written to the console.

Figure 7.2 Output of asynchronous example.

Binary Reading and Writing

Thus far, we've dealt primarily with text files. While it is true that text files make up the majority of business programming I/O tasks, you will often need to read and write files of a proprietary type. To do so, you will access them at the binary level. Suppose that you need to accept an Excel file streaming across the wire, chances are you will want to persist it to disk using a binary reader and writer. Or suppose you want to read image files and store them in your database. Again, a binary reader will make this operation go smoothly.

We have a number of options open to us for file I/O at the binary level. The principal ones include using the BinaryReader, BinaryWriter, and FileStream classes. The best thing is that, for the most part, you already know how to use these objects. BinaryReader and BinaryWriter are similar to StreamReader and StreamWriter, respectively. Like these classes, BinaryReader and BinaryWriter take an instance of a valid Stream object as a parameter of their constructor. The Stream object represents the backing store that is being read from or written to.

The BinaryReader class provides a number of read methods that allow us to access primitive data types from our file streams. Each read method returns the given data from the stream and advances the current position in the stream ahead of the returned data. The reader you're likely to use most often is ReadByte. This returns one byte of data from the stream and advances the current stream position to the next byte. When the end of the stream is reached, the exception, EndOfStreamException, is thrown by the method. Other read methods include ReadBytes, ReadString, ReadDecimal, and ReadBoolean to name a few.

Similarly, BinaryWriter provides a number of write methods for writing primitive data to a stream. Unlike BinaryReader, BinaryWriter exposes only one method, WriteByte, for executing binary writes. However, this method has a number of overloads that allow us to specify whether we are writing byte data or string, decimal, and so on. Calls to WriteByte write out the given data to the stream and advance its current position by the length of the data. Again, WriteByte(value as Byte) will be the most commonly used method.

The FileStream class also exposes the basic binary methods, ReadByte and WriteByte. ReadByte and WriteByte behave in the exact same manner as BinaryReader.ReadByte and BinaryWriter.WriteByte(value as Byte). It is often easier to simply use FileStream for all your basic needs; this is why it exists. Should you need additional functionality, then you will want to implement one or more of the binary classes.

Listing 7.7 provides an example of the BinaryReader and BinaryWriter classes. In the example, we use BinaryReader to read the contents of a bitmap file, one byte at a time. At the same time, we write each byte out to another file using BinaryWriter. The result is two identical files. Notice that to create both the reader and the writer we must first create a valid FileStream (or similar Stream derivation) for the instances to use as their backing.

LISTING 7.7 Binary Reading and Writing

Imports System.IO 

Module Module1 

   Sub Main() 

     'purpose: read a binary file and write contents to diff. file 

     'local scope 
     Dim fsRead As FileStream 
     Dim fsWrite As FileStream 
     Dim bRead As BinaryReader 
     Dim b As Byte 
     Dim bWrite As BinaryWriter 

     'check if read file exists 
     If Not File.Exists(path:="c:\test.bmp") Then 

        Console.WriteLine("File, test.bmp, not found") 

        'waite for user input 
        Console.WriteLine("Enter 's' to stop the application.") 
        Do While Console.ReadLine <> "s" : Loop 

        End 

     End If 

     'create a fileStream instance to pass to BinaryReader object 
     fsRead = New FileStream(path:="c:\test.bmp", mode:=FileMode.Open) 

     'check if write file exists 
     If File.Exists(path:="c:\test2.bmp") Then 

        'delete file 
        File.Delete(path:="c:\test2.bmp") 

     End If 

      'create a fileStream instance to pass to BinaryWriter object 
      fsWrite = New FileStream(path:="c:\test2.bmp", _ 
          mode:=FileMode.CreateNew, access:=FileAccess.Write) 

      'create binary writer instance 
      bWrite = New BinaryWriter(output:=fsWrite) 

      'create instance of binary reader 
      bRead = New BinaryReader(Input:=fsRead) 

      'set the file pointer to the start of the file 
      bRead.BaseStream.Seek(offset:=0, _ 
          origin:=SeekOrigin.Begin) 

      'loop until can no longer read bytes from file 
      Do While True 

         Try 

            'read next byte and advance reader 
            b = bRead.ReadByte 

         Catch 
            Exit Do 
         End Try 

         'write byte out 
         bWrite.Write(value:=b) 

      Loop 
 
      'close the reader 
      bRead.Close() 

      'close the writer 
      bWrite.Close() 

      'close the file streams 
      fsRead.Close() 
      fsWrite.Close() 

      'wait for the user to stop the console application 
      Console.WriteLine("Operation complete.") 
      Console.WriteLine("Enter 's' to stop the application.") 

      'loop until user presses s key 
      Do While Console.ReadLine <> "s" : Loop 
   End Sub 
End Module 

Suggestions for Further Exploration

• To create a stream whose backing is memory (and not disk), check out the MemoryStream class. This class is useful in that it can reduce your need for direct file I/O inside your application and provide you with a temporary buffer.

• Depending on your application, you can sometimes garner additional performance by implementing the BufferedStream class. Note that FileStream has internal buffering of its own and is often sufficient. Programming with BufferedStream is similar to the other stream classes we've discussed.

• To store data using isolated stores, check out the namespace System.IO.IsolatedStorage. Isolated stores are secure data compartments that are only accessible by the given user or code assembly. Data can also be isolated at the domain level and user data can travel with them using roaming profiles. For more information, check out the MSDN chapter at: Visual Studio .NET/.NET Framework/Programming with the .NET Framework/Working with I/O/Performing Isolated Storage Tasks.