问题描述

Code:

static void MultipleFilesToSingleFile(string dirPath, string filePattern, string destFile)
{
    string[] fileAry = Directory.GetFiles(dirPath, filePattern);

    Console.WriteLine("Total File Count : " + fileAry.Length);

    using (TextWriter tw = new StreamWriter(destFile, true))
    {
        foreach (string filePath in fileAry)
        {
            using (TextReader tr = new StreamReader(filePath))
            {
                tw.WriteLine(tr.ReadToEnd());
                tr.Close();
                tr.Dispose();
            }
            Console.WriteLine("File Processed : " + filePath);
        }

        tw.Close();
        tw.Dispose();
    }
}

I need to optimize this as its extremely slow: takes 3 minutes for 45 files of average size 40 — 50 Mb XML file.

Please note: 45 files of an average 45 MB is just one example, it can be n numbers of files of m size, where n is in thousands & m can be of average 128 Kb. In short, it can vary.

Could you please provide any views on optimization?

General answer

Why not just use the Stream.CopyTo(Stream destination) method?

private static void CombineMultipleFilesIntoSingleFile(string inputDirectoryPath, string inputFileNamePattern, string outputFilePath)
{
    string[] inputFilePaths = Directory.GetFiles(inputDirectoryPath, inputFileNamePattern);
    Console.WriteLine("Number of files: {0}.", inputFilePaths.Length);
    using (var outputStream = File.Create(outputFilePath))
    {
        foreach (var inputFilePath in inputFilePaths)
        {
            using (var inputStream = File.OpenRead(inputFilePath))
            {
                // Buffer size can be passed as the second argument.
                inputStream.CopyTo(outputStream);
            }
            Console.WriteLine("The file {0} has been processed.", inputFilePath);
        }
    }
}

Buffer size adjustment

Please, note that the mentioned method is overloaded.

There are two method overloads:

1. CopyTo(Stream destination).
2. CopyTo(Stream destination, int bufferSize).

The second method overload provides the buffer size adjustment through the bufferSize parameter.

这篇关于将多个文件合并为一个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！