File IO has shaped the computer industry more than any other technology. Doubt me? Consider that in 1978 AT&T decided to no longer share the source code of Unix with universities, upsetting professors like Andrew Tanenbaum. Andrew taught cources on operating systems and felt it was impossible to teach the subject without the source code. Why? Some concepts are harder to grasp in theory but easy in code, like threading. Other concepts are easy in theory, but hold a world of problems in implementation, like file IO. To solve this, Andrew created MINUX, a "minimal Unix" system for teaching. Later, a young lad in Finland named Linus started adding features to MINUX and jokingly referred to as "Linus' Minux", a name that stuck as Linux.
So what's so hard about file IO that requires a working example to learn? After all, you open a file, read or write some data, then close it - simple right? Consider what goes on when you call File.Open - the OS uses an index to figure out where the bits to the requested file are stored - and they may not be all stored in the same place. A magnetic disk begins to spin while a mechanical arm is positioned at the spot the OS designates. Electric current is generated from the variations in the magnetic field, passed over a controller to the CPU to make sense of, before passing it along to your application.
That's just the hardware, there may be more software involved in that operation. Virus scanners may kick in to make sure the file is safe, or network users may be half way though coping the file. Can your application handle the user snooping around with Notepad while you save some data? Nothing can tick off a user more than a corrupted save file, loosing hours of work in your application.
"Okay Mike, I get it - shut up and show me the code."
The methods I present here are designed to save and read objects as XML. I came up with these after going though numerous methods in the .Net Framework (it seems every base class has at least one file IO method these days), figuring out what worked best.
using System;
using System.Text;
using System.IO;
using System.Xml;
using System.Xml.Serialization;
using System.Threading;
public class IOTools {
public static void SerializeObject(String FileName, Type ObjectType, Object Data) {
Double timeout = 5000;
StringBuilder xmlData = new StringBuilder();
XmlSerializerNamespaces xsn = new XmlSerializerNamespaces();
xsn.Add(String.Empty, String.Empty);
StringWriterUTF8 stringWriter = new StringWriterUTF8(xmlData);
XmlTextWriter xmlWriter = new XmlTextWriter(stringWriter);
XmlSerializer xmlSerial = new XmlSerializer(ObjectType);
xmlSerial.Serialize(xmlWriter, Data, xsn);
xmlWriter.Close();
DateTime start = DateTime.Now;
Exception lastEx = null;
Boolean success = false;
while (DateTime.Now < start.AddMilliseconds(timeout)) {
try {
using (FileStream fs = File.Open(FileName, FileMode.Create, FileAccess.Write, FileShare.None)) {
StreamWriter writer = new StreamWriter(fs);
writer.Write(xmlData.ToString());
writer.Close();
success = true;
break;
}
}
catch (Exception ex) {
lastEx = ex;
Thread.Sleep(10);
}
}
if (!success) throw lastEx;
}
public static object DeserializeObject(String FileName, Type ObjectType) {
Double timeout = 5000;
String data = String.Empty;
DateTime start = DateTime.Now;
Exception lastEx = null;
Boolean success = false;
while (DateTime.Now < start.AddMilliseconds(timeout)) {
try {
using (FileStream fs = File.Open(FileName, FileMode.Open, FileAccess.Read, FileShare.Read)) {
StreamReader reader = new StreamReader(fs);
data = reader.ReadToEnd();
fs.Close();
success = true;
break;
}
}
catch (Exception ex) {
lastEx = ex;
Thread.Sleep(10);
}
}
if (!success) throw lastEx;
StringReader stringReader = new StringReader(data);
XmlTextReader xmlReader = new XmlTextReader(stringReader);
XmlSerializer xmlSerial = new XmlSerializer(ObjectType);
return xmlSerial.Deserialize(xmlReader);
}
public class StringWriterUTF8 : StringWriter {
public override Encoding Encoding {
get { return Encoding.UTF8; }
}
public StringWriterUTF8() : base() { }
public StringWriterUTF8(StringBuilder sb) : base(sb) { }
}
}
To minimize the chance of accessing the files while they are in use, the data is buffered in a string (or StringBuilder). The basic logic boils down to this: try to open the file for 5 seconds, and if you can't throw an exception. In testing I found a Thread.Sleep of just 10 milliseconds was enough that the process didn't show up in CPU usage (without Sleep, a locked file could cause 65% load for 5 seconds - ouch).
There is some extra XML work going on to save the files without namespaces and in UTF-8 (by default, all strings are UTF-16). Since my XML files aren't limited to being used by .Net apps this helps compatibility and the framework has a little known feature called "XML Namespace Hell" that's best to avoid. If you were certain your data would only be read by you or other .Net apps, I recommend replacing the XML with the binary serialization methods of System.Runtime.Serialization for better performance.
Using the methods looks like this:
// Saving as XML
MyCustomClass myClass = new MyCustomClass();
try { SerializeObject(@"C:\data\myclass.xml", typeof(MyCustomClass), myClass); }
catch { /* handle a failure */ }
// Loading from XML
MyCustomClass myClass = null;
try { myClass = (MyCustomClass)DeserializeObject(@"C:\data\myclass.xml", typeof(MyCustomClass)); }
catch { /* handle a failure */ }
These methods work for local drives and network shares alike, File.Open is pretty liberal with the possibilities of a full path. This is by no means asserted here as the final, full-proof way to handle routine file IO; if you have some field notes learned in battle please share them in the comment!
Posted By Mike On Friday, January 25, 2008
Filed under developer io |
Comments (3)
jim tollan
-
Wednesday, July 09, 2008
2:24:36 PM
Mike,
some good stuff up there. funnily enough stumbled upon this whilst searching for something else. thought i'd share my implementations of the above that use generics. i haven't refactored them to match your logic, but you'll get the idea. anyway, here we go:
#region xml serialisation methods
protected static bool SaveAsXml<T>(string path, List<T> objectList)
{
bool result = true;
FileStream fs = new FileStream(path, FileMode.Create);
try
{
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(typeof(List<T>));
x.Serialize(fs, objectList);
}
catch { result = false; } // do something here if error occurs
finally { fs.Close(); }
return result;
}
protected static List<T> LoadFromXml<T>(string path, List<T> objectList)
{
if (System.IO.File.Exists(path))
{
FileStream fo = new FileStream(path, FileMode.Open);
try
{
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(typeof(List<T>));
objectList = (List<T>)x.Deserialize(fo);
fo.Close();
}
catch { } // we'd put error handling code in here
finally { fo.Close(); }
}
return objectList;
}
#endregion
cheers -jim
jovan tomasevic
-
Friday, October 30, 2009
3:58:54 PM
this part
using (FileStream fs = File.Open(FileName, FileMode.Open, FileAccess.Read, FileShare.Read)) {
StreamReader reader = new StreamReader(fs);
data = reader.ReadToEnd();
fs.Close();
success = true;
break;
i try like this
FileStream str = new FileStream(path, FileMode.OpenOrCreate);
str.Flush();
ser.Serialize(str, doc, namespaces);
str.Flush();
str.Close();
but it's not good, files may be corrupted after serialization if new file is shorter...
anyway your code was very useful for me.
thanks for solution Mike
Mike
-
Friday, October 30, 2009
4:29:07 PM
You've mixed up the Serialize and DeSerialize method - the Serialize method uses the flag FileMode.Create, which will create a new file and truncate an existing one.
http://msdn.microsoft.com/en-us/library/system.io.filemode.aspx
