Directory size browser
This article shows how to build a responsive directory size browser application utilizing threading. The source code includes both C# and VB.Net
Introduction
The problem of this article has been touched several times, how to gather information about disk space usage for files and directories within a disk drive. However, I didn't find a suitable solution easily so I decided to make a small program for the task.
For the program I had few requirements it should satisfy:
- Visually show cumulative directory space usage
- Finding big files should be easy
- The program has elevated privileges in order to access all folders
- The user interface is responsive and informative while investigating the directories
So here are a few pictures for the outcome. Next we'll go through how the program was actually made.
Elevated privileges
Option 1: Manifest
Since I wanted to be able to search through all directories, the first thing is to set up the project so that it uses administrator privileges. Alternative 1 is ensure that administrator privileges are granted and if not, then the application won't run. This is done by adding a manifest file into the project. Simply select Add New Item... for the project and add an Application Manifest File.
The next step is to add proper trust info requirements. The newly created manifest file contains typical configuration options in comments so just select the requireAdministrator level like following
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
<security>
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
<requestedExecutionLevel level="requireAdministrator" uiAccess="false" />
</requestedPrivileges>
</security>
</trustInfo>
After this has been done when you start the program, Visual Studio will ask for administrator privileges. The same happens when you run the compiled exe.
The downside of this approach is that the privilege requirement is hardcoded into the application. The application itself would run without admin privileges, it just wouldn't be able to investigate all folders.
Option 2: Elevate privileges during the application startup
If you don't want to use hard coded elevated privileges, you can remove the manifest from the project. However, it still would be handy if the application could control if the privileges are going to be elevated.
The privileges of the process cannot be changed when the process has started, but we can always start a new process with more privileges. Based on this idea the startup method investigates if the process has admin privileges and if not, a question is asked if the privileges can be elevated. The code looks like this
[System.STAThread()]
public static void Main() {
DirectorySizes.Starter application;
DirectorySizes.DirectoryBrowser browserWindow;
System.Security.Principal.WindowsIdentity identity;
System.Security.Principal.WindowsPrincipal principal;
System.Windows.MessageBoxResult result;
System.Diagnostics.ProcessStartInfo adminProcess;
System.Windows.Input.Mouse.OverrideCursor = System.Windows.Input.Cursors.AppStarting;
{ // --- Alternative for using manifest for elevated privileges
// Check if admin rights are in place
identity = System.Security.Principal.WindowsIdentity.GetCurrent();
principal = new System.Security.Principal.WindowsPrincipal(identity);
// Ask for permission if not an admin
if (!principal.IsInRole(System.Security.Principal.WindowsBuiltInRole.Administrator)) {
result = System.Windows.MessageBox.Show(
"Can the application run in elevated mode in order to access all files?",
"Directory size browser",
System.Windows.MessageBoxButton.YesNo, System.Windows.MessageBoxImage.Question);
if (result == System.Windows.MessageBoxResult.Yes) {
// Re-run the application with administrator privileges
adminProcess = new System.Diagnostics.ProcessStartInfo();
adminProcess.UseShellExecute = true;
adminProcess.WorkingDirectory = System.Environment.CurrentDirectory;
adminProcess.FileName = System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName;
adminProcess.Verb = "runas";
try {
System.Diagnostics.Process.Start(adminProcess);
// quit after starting the new process
return;
} catch (System.Exception exception) {
System.Windows.MessageBox.Show(exception.Message, "Directory size browser",
System.Windows.MessageBoxButton.OK, System.Windows.MessageBoxImage.Exclamation);
return;
}
}
}
} // --- Alternative for using manifest for elevated privileges
application = new DirectorySizes.Starter();
browserWindow = new DirectorySizes.DirectoryBrowser();
application.Run(browserWindow);
}
<System.STAThread>
Public Shared Sub Main()
Dim application As DirectorySizes.Starter
Dim browserWindow As DirectorySizes.DirectoryBrowser
Dim identity As System.Security.Principal.WindowsIdentity
Dim principal As System.Security.Principal.WindowsPrincipal
Dim result As System.Windows.MessageBoxResult
Dim adminProcess As System.Diagnostics.ProcessStartInfo
System.Windows.Input.Mouse.OverrideCursor = System.Windows.Input.Cursors.AppStarting
' --- Alternative for using manifest for elevated privileges
' Check if admin rights are in place
identity = System.Security.Principal.WindowsIdentity.GetCurrent()
principal = New System.Security.Principal.WindowsPrincipal(identity)
' Ask for permission if not an admin
If (Not principal.IsInRole(System.Security.Principal.WindowsBuiltInRole.Administrator)) Then
result = System.Windows.MessageBox.Show(
"Can the application run in elevated mode in order to access all files?",
"Directory size browser",
System.Windows.MessageBoxButton.YesNo,
System.Windows.MessageBoxImage.Question)
If (result = System.Windows.MessageBoxResult.Yes) Then
adminProcess = New System.Diagnostics.ProcessStartInfo()
adminProcess.UseShellExecute = True
adminProcess.WorkingDirectory = System.Environment.CurrentDirectory
adminProcess.FileName = System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName
adminProcess.Verb = "runas"
Try
System.Diagnostics.Process.Start(adminProcess)
Return
Catch exception As System.Exception
System.Windows.MessageBox.Show(exception.Message, "Directory size browser",
System.Windows.MessageBoxButton.OK,
System.Windows.MessageBoxImage.Exclamation)
Return
End Try
End If
End If
' // --- Alternative for using manifest for elevated privileges
application = New DirectorySizes.Starter()
browserWindow = New DirectorySizes.DirectoryBrowser()
application.Run(browserWindow)
End Sub
If the user accepts the elevation a new process is started with runas verb. This process is then let to end so the user interface starts in the newly created process.
The downside of this approach is that the process changes. If the application is simply run this doesn't matter but if you're debugging the application then the original process ends and the Visual Studio debugger closes. So in order to debug the application with elevated privileges, you need to attach the debugger to the new process.
Because of this behaviour I decided to leave the manifest in place in the download. So if you want to try this approach, comment out the manifest.
The program itself
The main logic
The application consists of few main classes. These are:
DirectoryBrowserwindow, the user interface.DirectoryHelper, a static class containing all the logic for gathering the information.DirectoryDetailandFileDetail, these classes hold the data.
The data gather is done in a recursive method called ListFiles. Let's have a look at it in whole and then have a closer look at it part by part. So the method looks like this
/// <summary>
/// Adds recursively files and directories to hashsets
/// </summary>
/// <param name="directory">Directory to gather data from</param>
/// <returns>Directory details</returns>
private static DirectoryDetail ListFiles(DirectoryDetail thisDirectoryDetail) {
DirectoryDetail subDirectoryDetail;
System.IO.FileInfo fileInfo;
// Exit if stop is requested
lock (DirectoryHelper._lockObject) {
if (DirectoryHelper._stopRequested) {
return thisDirectoryDetail;
}
}
RaiseStatusUpdate(string.Format("Analyzing {0}", DirectoryHelper.ShortenPath(thisDirectoryDetail.Path)));
//List files in this directory
try {
// Loop through child directories
foreach (string subDirectory
in System.IO.Directory.EnumerateDirectories(thisDirectoryDetail.Path).OrderBy(x => x)) {
subDirectoryDetail = ListFiles(new DirectoryDetail(subDirectory,
thisDirectoryDetail.Depth + 1,
thisDirectoryDetail));
thisDirectoryDetail.CumulativeSize += subDirectoryDetail.CumulativeSize;
thisDirectoryDetail.CumulativeNumberOfFiles += subDirectoryDetail.CumulativeNumberOfFiles;
thisDirectoryDetail.SubDirectoryDetails.Add(subDirectoryDetail);
// Break if stop is requested
lock (DirectoryHelper._lockObject) {
if (DirectoryHelper._stopRequested) {
break;
}
}
}
if (!DirectoryHelper._stopRequested) {
// List files in this directory
foreach (string file
in System.IO.Directory.EnumerateFiles(thisDirectoryDetail.Path, "*.*",
System.IO.SearchOption.TopDirectoryOnly)) {
fileInfo = new System.IO.FileInfo(file);
lock (DirectoryHelper._lockObject) {
FileDetails.Add(new FileDetail() {
Name = fileInfo.Name,
Path = fileInfo.DirectoryName,
Size = fileInfo.Length,
LastAccessed = fileInfo.LastAccessTime,
Extension = fileInfo.Extension,
DirectoryDetail = thisDirectoryDetail
});
}
thisDirectoryDetail.CumulativeSize += fileInfo.Length;
thisDirectoryDetail.Size += fileInfo.Length;
thisDirectoryDetail.NumberOfFiles++;
thisDirectoryDetail.CumulativeNumberOfFiles++;
DirectoryHelper.OverallFileCount++;
}
}
// add this directory to the collection
lock (DirectoryHelper._lockObject) {
DirectoryDetails.Add(thisDirectoryDetail);
}
DirectoryHelper.OverallDirectoryCount++;
DirectoryHelper.RaiseCountersChanged();
} catch (System.UnauthorizedAccessException exception) {
// Listing files in the directory not allowed so ignore this directory
lock (DirectoryHelper._lockObject) {
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path);
}
DirectoryHelper.RaiseDirectorySkipped(string.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message));
} catch (System.IO.PathTooLongException exception) {
// Path is too long
lock (DirectoryHelper._lockObject) {
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path);
}
DirectoryHelper.RaiseDirectorySkipped(string.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message));
}
if (thisDirectoryDetail.Depth == 1) {
if (DirectoryComplete != null) {
DirectoryComplete(null, thisDirectoryDetail);
}
}
return thisDirectoryDetail;
}
' Adds recursively files and directories to hashsets
Private Function ListFiles(thisDirectoryDetail As DirectoryDetail) As DirectoryDetail
Dim subDirectoryDetail As DirectoryDetail
Dim fileInfo As System.IO.FileInfo
' Exit if stop is requested
SyncLock (DirectoryHelper._lockObject)
If (DirectoryHelper._stopRequested) Then
Return thisDirectoryDetail
End If
End SyncLock
RaiseStatusUpdate(String.Format("Analyzing {0}", DirectoryHelper.ShortenPath(thisDirectoryDetail.Path)))
' List files in this directory
Try
' Loop through child directories
For Each subDirectory As String
In System.IO.Directory.EnumerateDirectories(thisDirectoryDetail.Path).OrderBy(Function(x) x)
subDirectoryDetail = ListFiles(New DirectoryDetail(subDirectory,
thisDirectoryDetail.Depth + 1,
thisDirectoryDetail))
thisDirectoryDetail.CumulativeSize += subDirectoryDetail.CumulativeSize
thisDirectoryDetail.CumulativeNumberOfFiles += subDirectoryDetail.CumulativeNumberOfFiles
thisDirectoryDetail.SubDirectoryDetails.Add(subDirectoryDetail)
' Break if stop is requested
SyncLock (DirectoryHelper._lockObject)
If (DirectoryHelper._stopRequested) Then
Exit For
End If
End SyncLock
Next subDirectory
If (Not DirectoryHelper._stopRequested) Then
' List files in this directory
For Each file As String
In System.IO.Directory.EnumerateFiles(thisDirectoryDetail.Path, "*.*",
System.IO.SearchOption.TopDirectoryOnly)
fileInfo = New System.IO.FileInfo(file)
SyncLock (DirectoryHelper._lockObject)
FileDetails.Add(New FileDetail() With {
.Name = fileInfo.Name,
.Path = fileInfo.DirectoryName,
.Size = fileInfo.Length,
.LastAccessed = fileInfo.LastAccessTime,
.Extension = fileInfo.Extension,
.DirectoryDetail = thisDirectoryDetail
})
End SyncLock
thisDirectoryDetail.CumulativeSize += fileInfo.Length
thisDirectoryDetail.Size += fileInfo.Length
thisDirectoryDetail.NumberOfFiles += 1
thisDirectoryDetail.CumulativeNumberOfFiles += 1
DirectoryHelper.OverallFileCount += 1
DirectoryHelper.RaiseCountersChanged()
Next file
End If
' add this directory to the collection
SyncLock (DirectoryHelper._lockObject)
DirectoryDetails.Add(thisDirectoryDetail)
End SyncLock
DirectoryHelper.OverallDirectoryCount += 1
DirectoryHelper.RaiseCountersChanged()
Catch exception As System.UnauthorizedAccessException
' Listing files in the directory not allowed so ignore this directory
SyncLock (DirectoryHelper._lockObject)
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path)
End SyncLock
DirectoryHelper.RaiseDirectorySkipped(String.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message))
Catch exception As System.IO.PathTooLongException
' Path is too long
SyncLock (DirectoryHelper._lockObject)
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path)
End SyncLock
DirectoryHelper.RaiseDirectorySkipped(String.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message))
End Try
If (thisDirectoryDetail.Depth = 1) Then
RaiseEvent DirectoryComplete(Nothing, thisDirectoryDetail)
End If
Return thisDirectoryDetail
End Function
At this point let's skip the Raise... methods and the locking. We'll get back to those. What the method does is, it receives a directory as a parameter, loops through its sub directories and for each sub directory it calls ListFiles method again to achieve recursion:
// Loop through child directories
foreach (string subDirectory
in System.IO.Directory.EnumerateDirectories(thisDirectoryDetail.Path).OrderBy(x => x)) {
subDirectoryDetail = ListFiles(new DirectoryDetail(subDirectory,
thisDirectoryDetail.Depth + 1,
thisDirectoryDetail));
thisDirectoryDetail.CumulativeSize += subDirectoryDetail.CumulativeSize;
thisDirectoryDetail.CumulativeNumberOfFiles += subDirectoryDetail.CumulativeNumberOfFiles;
thisDirectoryDetail.SubDirectoryDetails.Add(subDirectoryDetail);
// Break if stop is requested
lock (DirectoryHelper._lockObject) {
if (DirectoryHelper._stopRequested) {
break;
}
}
}
' Loop through child directories
For Each subDirectory As String In System.IO.Directory.EnumerateDirectories(thisDirectoryDetail.Path).OrderBy(Function(x) x)
subDirectoryDetail = ListFiles(New DirectoryDetail(subDirectory,
thisDirectoryDetail.Depth + 1,
thisDirectoryDetail))
thisDirectoryDetail.CumulativeSize += subDirectoryDetail.CumulativeSize
thisDirectoryDetail.CumulativeNumberOfFiles += subDirectoryDetail.CumulativeNumberOfFiles
thisDirectoryDetail.SubDirectoryDetails.Add(subDirectoryDetail)
' Break if stop is requested
SyncLock (DirectoryHelper._lockObject)
If (DirectoryHelper._stopRequested) Then
Exit For
End If
End SyncLock
Next subDirectory
When the recursion ends the ListFiles method returns a DirectoryDetail object which is filled with the data gathered for that directory and it's subdirectories. When the execution returns from the recursion the CumulativeSize and CumulativeNumberOfFiles for the directory at hand are incremented based on the values returned from the recursion. This helps to investigate the directories based on size later on. The next step gathers information about the files in current directory
// List files in this directory
foreach (string file in System.IO.Directory.EnumerateFiles(thisDirectoryDetail.Path, "*.*",
System.IO.SearchOption.TopDirectoryOnly)) {
fileInfo = new System.IO.FileInfo(file);
lock (DirectoryHelper._lockObject) {
FileDetails.Add(new FileDetail() {
Name = fileInfo.Name,
Path = fileInfo.DirectoryName,
Size = fileInfo.Length,
LastAccessed = fileInfo.LastAccessTime,
Extension = fileInfo.Extension,
DirectoryDetail = thisDirectoryDetail
});
}
thisDirectoryDetail.CumulativeSize += fileInfo.Length;
thisDirectoryDetail.Size += fileInfo.Length;
thisDirectoryDetail.NumberOfFiles++;
thisDirectoryDetail.CumulativeNumberOfFiles++;
DirectoryHelper.OverallFileCount++;
}
lock (DirectoryHelper._lockObject) {
DirectoryDetails.Add(thisDirectoryDetail);
}
DirectoryHelper.OverallDirectoryCount++;
DirectoryHelper.RaiseCountersChanged();
' List files in this directory
For Each file As String In System.IO.Directory.EnumerateFiles(thisDirectoryDetail.Path, "*.*",
System.IO.SearchOption.TopDirectoryOnly)
fileInfo = New System.IO.FileInfo(file)
SyncLock (DirectoryHelper._lockObject)
FileDetails.Add(New FileDetail() With {
.Name = fileInfo.Name,
.Path = fileInfo.DirectoryName,
.Size = fileInfo.Length,
.LastAccessed = fileInfo.LastAccessTime,
.Extension = fileInfo.Extension,
.DirectoryDetail = thisDirectoryDetail
})
End SyncLock
thisDirectoryDetail.CumulativeSize += fileInfo.Length
thisDirectoryDetail.Size += fileInfo.Length
thisDirectoryDetail.NumberOfFiles += 1
thisDirectoryDetail.CumulativeNumberOfFiles += 1
DirectoryHelper.OverallFileCount += 1
DirectoryHelper.RaiseCountersChanged()
Next file
So this is a simple loop to go through all files in the directory and for each file found a new FileDetail object is created. Also the cumulative counters for the directory are incremented.
As you can see both DirectoryDetail and FileDetail objects are added to separate collections (FileDetails and DirectoryDetails) during the process. Even though only DirectoryDetails collection would be needed if FileDetails would be included for each directory, it's far better to have a single collection containing all the files when we want to list and sort the files regardless of the directory.
Of course things can go wrong and actually they will. Because of this there's a catch block for UnauthorizedAccessException. Even though we are using administrator privileges, some of the directories will cause this exception. Mainly these are junction points in NTFS. For more information, visit NTFS reparse point. When this type of directory is encountered, it's listed in the SkippedDirectories collection which then again is shown to the user using an event. The same handling applies if a PathTooLongException is raised.
} catch (System.UnauthorizedAccessException exception) {
// Listing files in the directory not allowed so ignore this directory
lock (DirectoryHelper._lockObject) {
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path);
}
DirectoryHelper.RaiseDirectorySkipped(string.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message));
} catch (System.IO.PathTooLongException exception) {
// Path is too long
lock (DirectoryHelper._lockObject) {
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path);
}
DirectoryHelper.RaiseDirectorySkipped(string.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message));
}
Catch exception As System.UnauthorizedAccessException
' Listing files in the directory not allowed so ignore this directory
SyncLock (DirectoryHelper._lockObject)
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path)
End SyncLock
DirectoryHelper.RaiseDirectorySkipped(String.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message))
Catch exception As System.IO.PathTooLongException
' Path is too long
SyncLock (DirectoryHelper._lockObject)
DirectoryHelper.SkippedDirectories.Add(thisDirectoryDetail.Path)
End SyncLock
DirectoryHelper.RaiseDirectorySkipped(String.Format("Skipped {0}, reason: {1}",
thisDirectoryDetail.Path, exception.Message))
End Try
After all the data gather is done, the method returns.
if (thisDirectoryDetail.Depth == 1) {
if (DirectoryComplete != null) {
DirectoryComplete(null, thisDirectoryDetail);
}
}
return thisDirectoryDetail;
If (thisDirectoryDetail.Depth = 1) Then
RaiseEvent DirectoryComplete(Nothing, thisDirectoryDetail)
End If
Return thisDirectoryDetail
Depending on the recursion the filled DirectoryDetail object will be returned to the previous level of recursion or to the original caller.
Gathering the data asynchronously
As said in the beginning I wanted the user interface to be responsive even if the data gather is in progress. Actually I wanted to be able to investigate a directory as soon as the information for it and its sub directories is gathered. This means that the gathering needs to be done asynchronously. As this is a WPF application with framework 4 or above a Task object will be used.
The GatherData which initiates the recursion looks like this
/// <summary>
/// Collects the data for a drive
/// </summary>
/// <param name="drive">Drive to investigate</param>
/// <returns>True if succesful</returns>
private static bool GatherData(System.IO.DriveInfo driveInfo) {
DirectoryHelper.RaiseGatherInProgressChanges(true);
DirectoryHelper.ListFiles(new DirectoryDetail(driveInfo.Name, 0,
driveInfo.TotalSize - driveInfo.AvailableFreeSpace));
DirectoryHelper.RaiseStatusUpdate("Calculating statistics...");
DirectoryHelper.CalculateStatistics();
DirectoryHelper.RaiseStatusUpdate("Idle");
DirectoryHelper.RaiseGatherInProgressChanges(false);
return true;
}
' Collects the data for a drive
Private Function GatherData(driveInfo As System.IO.DriveInfo) As Boolean
DirectoryHelper.RaiseGatherInProgressChanges(True)
DirectoryHelper.ListFiles(New DirectoryDetail(driveInfo.Name, 0,
driveInfo.TotalSize - driveInfo.AvailableFreeSpace))
DirectoryHelper.RaiseStatusUpdate("Calculating statistics...")
DirectoryHelper.CalculateStatistics()
DirectoryHelper.RaiseStatusUpdate("Idle")
DirectoryHelper.RaiseGatherInProgressChanges(False)
Return True
End Function
It raises an event to inform that the search is in progress then starts the recursion. But the interesting part is how this method is called.
/// <summary>
/// Starts the data gathering process
/// </summary>
/// <param name="drive"></param>
/// <returns></returns>
internal static bool StartDataGathering(System.IO.DriveInfo driveInfo) {
DirectoryHelper.FileDetails.Clear();
DirectoryHelper.DirectoryDetails.Clear();
DirectoryHelper.ExtensionDetails.Clear();
DirectoryHelper.SkippedDirectories.Clear();
DirectoryHelper.OverallDirectoryCount = 0;
DirectoryHelper.OverallFileCount = 0;
DirectoryHelper._stopRequested = false;
DirectoryHelper._gatherTask = new System.Threading.Tasks.Task(
() => { GatherData(driveInfo); },
System.Threading.Tasks.TaskCreationOptions.LongRunning);
DirectoryHelper._gatherTask.Start();
return true;
}
' Starts the data gathering process
Friend Function StartDataGathering(driveInfo As System.IO.DriveInfo) As Boolean
DirectoryHelper.FileDetails.Clear()
DirectoryHelper.DirectoryDetails.Clear()
DirectoryHelper.ExtensionDetails.Clear()
DirectoryHelper.SkippedDirectories.Clear()
DirectoryHelper.OverallDirectoryCount = 0
DirectoryHelper.OverallFileCount = 0
DirectoryHelper._stopRequested = False
DirectoryHelper._gatherTask = New System.Threading.Tasks.Task(
Function() GatherData(driveInfo),
System.Threading.Tasks.TaskCreationOptions.LongRunning)
DirectoryHelper._gatherTask.Start()
Return True
End Function
The beginning of the method is just clearing variables in case this isn't the first run. The Task constructor is used to tell what code should be run as the action for this task. In this case the GatherData method is called. Also the task is informed that the code is going to be long running so there's no point of doing fine grained scheduling.
When the Start method is called the DataGather method starts in its own separate thread and this method continues on in the UI thread. So now we have two different threads working, one running the UI and the other one collecting data for the directories.
Getting information from the other thread
Now when the other thread is working and making progress it sure would be nice to know that something is actually happening. I decided to inform the user about the directory currently under investigation and how many files or folders have been found so far. This information is sent to the user interface using events, so nothing very special here. The class has few static events such as
/// <summary>
/// Event used to send information about the gather process
/// </summary>
internal static event System.EventHandler<string> StatusUpdate;
/// <summary>
/// Event used to inform that the overall statistics have been changed
/// </summary>
internal static event System.EventHandler CountersChanged;
' Event used to send information about the gather process
Friend Event StatusUpdate As System.EventHandler(Of String)
' Event used to inform that the overall counters have been changed
Friend Event CountersChanged As System.EventHandler
And when the window is created it wires these events in a normal way.
// Wire the events
DirectoryHelper.StatusUpdate += DirectoryHelper_StatusUpdate;
DirectoryHelper.CountersChanged += DirectoryHelper_CountersChanged;
' Wire the events
AddHandler DirectoryHelper.StatusUpdate, AddressOf DirectoryHelper_StatusUpdate
AddHandler DirectoryHelper.CountersChanged, AddressOf DirectoryHelper_CountersChanged
However, if these events would try to directly update the status item to contain the directory name passed as a parameter or modify any other user interface object an InvalidOperationException would be raised
An exception of type 'System.InvalidOperationException' occurred in WindowsBase.dll but was not handled in user code
Additional information: The calling thread cannot access this object because a different thread owns it.
Remember, the UI runs in a separate thread than the data gatherer which sent the event. In order to update the window we need to change the context back to the UI thread. This is done using the dispatcher that is created by the UI thread and calling the BeginInvoke method to execute a piece of code that does the UI operations.
So the status update is as simple as this
/// <summary>
/// Updates the status bar
/// </summary>
/// <param name="state"></param>
private void UpdateStatus(string state) {
this.Status.Content = state;
}
' Updates the status bar
Private Sub UpdateStatus(state As String)
Me.Status.Content = state
End Sub
and the call in the event handler to execute UpdateStatus method is like the following
private void DirectoryHelper_StatusUpdate(object sender, string e) {
this.Status.Dispatcher.BeginInvoke((System.Action)(() => { UpdateStatus(e); }));
}
Private Sub DirectoryHelper_StatusUpdate(sender As Object, e As String)
Me.Status.Dispatcher.BeginInvoke(Sub() UpdateStatus(e))
End Sub
This needs some explanation. This overload of the BeginInvoke is called with an Action delegate or Sub in VB. This anonymous delegate executes the UpdateStatus method. So in the code above there's no predefined delegate. Another option would be to define a delegate, for example
private delegate void UpdateGatherCountersDelegate(bool forceUpdate);
Private Delegate Sub UpdateGatherCountersDelegate(forceUpdate As Boolean)
Then define the actual method having the same method signature as in the delegate defintion
/// <summary>
/// Updates the counters
/// </summary>
void UpdateGatherCounters(bool forceUpdate) {
if (forceUpdate || System.DateTime.Now.Subtract(this._lastCountersUpdate).TotalMilliseconds > 500) {
this.CountInfo.Content = string.Format("{0} directories, {1} files",
DirectoryHelper.OverallDirectoryCount.ToString("N0"),
DirectoryHelper.OverallFileCount.ToString("N0"));
this._lastCountersUpdate = System.DateTime.Now;
}
}
' Updates the counters
Sub UpdateGatherCounters(forceUpdate As Boolean)
If (forceUpdate Or System.DateTime.Now.Subtract(Me._lastCountersUpdate).TotalMilliseconds > 500) Then
Me.CountInfo.Content = String.Format("{0} directories, {1} files",
DirectoryHelper.OverallDirectoryCount.ToString("N0"),
DirectoryHelper.OverallFileCount.ToString("N0"))
Me._lastCountersUpdate = System.DateTime.Now
End If
End Sub
And then use the BeginInvoke with the predefined delegate like this
void DirectoryHelper_CountersChanged(object sender, System.EventArgs e) {
this.CountInfo.Dispatcher.BeginInvoke(new UpdateGatherCountersDelegate(UpdateGatherCounters), false);
}
Private Sub DirectoryHelper_StatusUpdate(sender As Object, e As String)
Me.Status.Dispatcher.BeginInvoke(Sub() UpdateStatus(e))
End Sub
Accessing the collections during the data gather
Okay, now we update the UI while the data gather is in progress. One of the events, DirectoryComplete is raised whenever the data gather for a root folder is complete. Within this event the folder is added to the directories list and the user can start investigating it. The user can expand the directory and see the sub directories. User can also select the directory and see list of the files in that specific folder and 100 biggest files in that path.
The directory is added using the AddRootNode method
/// <summary>
/// Used to add root folders
/// </summary>
/// <param name="directoryDetail">Directory to add</param>
private void AddRootNode(DirectoryDetail directoryDetail) {
AddDirectoryNode(this.DirectoryTree.Items, directoryDetail);
}
' Used to add root folders
Private Sub AddRootNode(directoryDetail As DirectoryDetail)
AddDirectoryNode(Me.DirectoryTree.Items, directoryDetail)
End Sub
This method simply calls a common method to add a directory node since the same method is used when a directory node is expanded. So adding a node looks like this​
/// <summary>
/// Adds a directory node to the specified items collection
/// </summary>
/// <param name="parentItemCollection">Items collection of the parent directory</param>
/// <param name="directoryDetail">Directory to add</param>
/// <returns>True if succesful</returns>
private bool AddDirectoryNode(System.Windows.Controls.ItemCollection parentItemCollection,
DirectoryDetail directoryDetail) {
System.Windows.Controls.TreeViewItem treeViewItem;
System.Windows.Controls.StackPanel stackPanel;
// Create the stackpanel and it's content
stackPanel = new System.Windows.Controls.StackPanel();
stackPanel.Orientation = System.Windows.Controls.Orientation.Horizontal;
// Content
stackPanel.Children.Add(
this.CreateProgressBar("Cumulative percentage from total used space {0}% ({1}))",
directoryDetail.CumulativeSizePercentage,
directoryDetail.FormattedCumulativeBytes));
stackPanel.Children.Add(new System.Windows.Controls.TextBlock() {
Text = directoryDetail.DirectoryName });
// Create the treeview item
treeViewItem = new System.Windows.Controls.TreeViewItem();
treeViewItem.Tag = directoryDetail;
treeViewItem.Header = stackPanel;
treeViewItem.Expanded += tvi_Expanded;
// If this directory contains subdirectories, add a placeholder
if (directoryDetail.SubDirectoryDetails.Count() > 0) {
treeViewItem.Items.Add(new System.Windows.Controls.TreeViewItem() { Name = "placeholder" });
}
// Add the treeview item into the items collection
parentItemCollection.Add(treeViewItem);
return true;
}
' Adds a directory node to the specified items collection
Private Function AddDirectoryNode(parentItemCollection As System.Windows.Controls.ItemCollection, directoryDetail As DirectoryDetail) As Boolean
Dim treeViewItem As System.Windows.Controls.TreeViewItem
Dim stackPanel As System.Windows.Controls.StackPanel
' Create the stackpanel and it's content
stackPanel = New System.Windows.Controls.StackPanel()
stackPanel.Orientation = System.Windows.Controls.Orientation.Horizontal
' Content
stackPanel.Children.Add(Me.CreateProgressBar("Cumulative percentage from total used space {0}% ({1}))",
directoryDetail.CumulativeSizePercentage,
directoryDetail.FormattedCumulativeBytes))
stackPanel.Children.Add(New System.Windows.Controls.TextBlock() With {
.Text = directoryDetail.DirectoryName})
' Create the treeview item
treeViewItem = New System.Windows.Controls.TreeViewItem()
treeViewItem.Tag = directoryDetail
treeViewItem.Header = stackPanel
AddHandler treeViewItem.Expanded, AddressOf tvi_Expanded
' If this directory contains subdirectories, add a placeholder
If (directoryDetail.SubDirectoryDetails.Count() > 0) Then
treeViewItem.Items.Add(New System.Windows.Controls.TreeViewItem() With {.Name = "placeholder"})
End If
' Add the treeview item into the items collection
parentItemCollection.Add(treeViewItem)
Return True
End Function
The idea is that each directory node will show the cumulative percentage of disk space usage in a ProgressBar and the directory name. These controls are placed inside a StackPanel which then again is set as the header of the TreeViewItem.
So far so good, but if the TreeViewItem is selected, the program lists the content of the directory like this
/// <summary>
/// Populates the file list for a directory in descending order based on the file sizes
/// </summary>
/// <param name="tvi">Directory to populate files for</param>
private void ListDirectoryFiles(System.Windows.Controls.TreeViewItem tvi) {
DirectoryDetail directoryDetail;
this.FileList.ItemsSource = null;
this.Top100FileList.ItemsSource = null;
if (tvi != null) {
directoryDetail = (DirectoryDetail)tvi.Tag;
this.FileList.ItemsSource = DirectoryHelper.FilesInDirectory(directoryDetail);
this.Top100FileList.ItemsSource = DirectoryHelper.BiggestFilesInPath(directoryDetail, 100);
}
}
' Populates the file list for a directory in descending order based on the file sizes
Private Sub ListDirectoryFiles(tvi As System.Windows.Controls.TreeViewItem)
Dim directoryDetail As DirectoryDetail
Me.FileList.ItemsSource = Nothing
Me.Top100FileList.ItemsSource = Nothing
If (Not tvi Is Nothing) Then
directoryDetail = CType(tvi.Tag, DirectoryDetail)
Me.FileList.ItemsSource = DirectoryHelper.FilesInDirectory(directoryDetail)
Me.Top100FileList.ItemsSource = DirectoryHelper.BiggestFilesInPath(directoryDetail, 100)
End If
End Sub
And the file list is fetched in the following code
/// <summary>
/// Lists all the files in a directory sorted by size in descending order
/// </summary>
/// <param name="directoryDetail"></param>
/// <returns></returns>
internal static System.Collections.Generic.List<FileDetail> FilesInDirectory(
DirectoryDetail directoryDetail) {
System.Collections.Generic.List<FileDetail> fileList;
lock (DirectoryHelper._lockObject) {
fileList = DirectoryHelper.FileDetails
.Where(x => x.DirectoryDetail == directoryDetail)
.OrderByDescending(x => x.Size).ToList();
}
return fileList;
}
' Lists all the files in a directory sorted by size in descending order
Friend Function FilesInDirectory(directoryDetail As DirectoryDetail) As System.Collections.Generic.List(Of FileDetail)
Dim fileList As System.Collections.Generic.List(Of FileDetail)
SyncLock (DirectoryHelper._lockObject)
fileList = DirectoryHelper.FileDetails.Where(Function(x) x.DirectoryDetail Is directoryDetail)
.OrderByDescending(Function(x) x.Size).ToList()
End SyncLock
Return fileList
End Function
Let's have a bit closer look on this. If the lock statement would commented out and the list is populated at the same time when the data gather is in progress, you have roughly 100% chance of hitting the following error
An unhandled exception of type 'System.InvalidOperationException' occurred in System.Core.dll
Additional information: Collection was modified; enumeration operation may not execute.
What just would have happened is that when we tried to populate the file list the other thread added data into the same collection simultaneously. So the enumeration failed because the content of the collection was changed.
In order to prevent this from happening we need to have a mechanism that allows only one thread to enumerate or modify the collection at a time. This means locking.
The lock statement ensures that only one thread can enter a critical section of code at a time. If another thread tries to lock the same object it'll have to wait until the lock is freed. In the beginning of the article we saw how a new DirectoryDetail was added to the collection like this
lock (DirectoryHelper._lockObject) {
DirectoryDetails.Add(thisDirectoryDetail);
}
SyncLock (DirectoryHelper._lockObject)
DirectoryDetails.Add(thisDirectoryDetail)
End SyncLock
So now when we read the collection we try to lock the same _lockObject. This ensures that enumeration and modifications are not done at the same time.
However, that's not the entire story. You may wonder why the enumeration code has a ToList() method call in the end of the statement. If the code would return only an IOrderedEnumerable<FileDetail> collection for the caller the actual enumeration would happen outside the scope of the lock statement.
In other words when this collection would return to the WPF control, it would try to enumerate the collection and the exact same problem would be hit; Collection was modified; enumeration operation may not execute. Because of this the code creates a copy, the list, inside the scope of the lock and after that it's safe to return the list and let the WPF control loop through the collection.
Stopping the execution
The last detail is how to stop the data gather process before all directories have been investigated. This again involves locking. There are few, controlled places where the execution is stopped if needed. Just before a directory investigation is done or after a sub directory is processed.
The code simply investigates if a stop is requested using a static variable, like this
// Break if stop is requested
lock (DirectoryHelper._lockObject) {
if (DirectoryHelper._stopRequested) {
break;
}
}
' Break if stop is requested
SyncLock (DirectoryHelper._lockObject)
If (DirectoryHelper._stopRequested) Then
Exit For
End If
End SyncLock
If the stop button is pressed during the data gather, the _stopRequest is changed in the StopDataGathering method
/// <summary>
/// Stops the data gathering process
/// </summary>
/// <param name="drive"></param>
/// <returns></returns>
internal static bool StopDataGathering() {
if (DirectoryHelper._gatherTask.Status != System.Threading.Tasks.TaskStatus.Running) {
return true;
}
lock (DirectoryHelper._lockObject) {
DirectoryHelper._stopRequested = true;
}
DirectoryHelper._gatherTask.Wait();
return true;
}
' Stops the data gathering process
Friend Function StopDataGathering() As Boolean
If (DirectoryHelper._gatherTask.Status <> System.Threading.Tasks.TaskStatus.Running) Then
Return True
End If
SyncLock (DirectoryHelper._lockObject)
DirectoryHelper._stopRequested = True
End SyncLock
DirectoryHelper._gatherTask.Wait()
Return True
End Function
After the request status has been changed the UI thread calls Wait method for the task so that the UI thread is stopped until the data gather thread finishes.
So that covers the main parts of the program. There are a lot of details in the project if you want to investigate it but this should give the big picture how the application was made.
Oh, by the way, if you double click a file in any of the file lists it should open the containing folder for you.
Hopefully you find this useful and of course all comments and suggestions are more than welcome.
References
Here are some references that should be useful when going through the code
History
- 10th February, 2015: Article created.
- 13th February, 2015:
- Added catch for PathTooLongException and reformatted the skipped directory info.
- Added the ability to browse mapped network drives.
- Added more frequent counter change in case of large directories.
- 16th February, 2015: Alternative option for elevated privileges added.
- 17th February, 2015: VB version added.