Writing huge amounts of text to a textbox

I am writing a log of lots and lots of formatted text to a textbox in a .net windows form app.

It is slow once the data gets over a few megs. Since I am appending the string has to be reallocated every time right? I only need to set the value to the text box once, but in my code I am doing line+=data tens of thousands of times.

Is there a faster way to do this? Maybe a different control? Is there a linked list string type I can use?

Thanks.

From stackoverflow

• Build your String together with a StringBuilder, then convert it to a String using toString(), and assign this to the textbox.

  Jimmy Hoffa : Maybe StringBuffer is a class I've never heard of, but I'm wagering it's more likely you meant StringBuilder

  Marc : Wrong language, `StringBuffer` is java. =)

  Frank : Yes, you are correct, I will edit the answer.

• StringBuilder will not help if the text box is added to incrementally, like log output for example.

  But, if the above is true and if your updates are frequent enough it may behoove you to cache some number of updates and then append them in one step (rather than appending constantly). That would save you many string reallocations... and then StringBuilder would be helpful.

  Notes:

  1. Create a class-scoped StringBuilder member (_sb)
  2. Start a timer (or use a counter)
  3. Append text updates to _sb
  4. When timer ticks or certain counter reached reset and append to text box
  5. restart process from #1
  Marc : While your answer is definitely relevant, I think the statement *I only need to set the value to the text box once, but in my code I am doing line+=data tens of thousands of times.* means it won't be an answer to his specific problem.

  Steven Sudit : @Marc: Actually, Paul's answer is very good. One thread locks the SB, appends to it, unlocks. It does this many, many times. The UI thread can poll some small number of times per second, lock the SB, get the ToString and Clear it. This way, the UI update frequency is completely independent of the concatenation frequency.

• No one has mentioned virtualization yet, which is really the only way to provide predictable performance for massive volumes of data. Even using a StringBuilder and converting it to a string every half a second will be very slow once the log gets large enough.

  With data virtualization, you would only hold the necessary data in memory (i.e. what the user can see, and perhaps a little more on either side) whilst the rest would be stored on disk. Old data would "roll out" of memory as new data comes in to replace it.

  In order to make the TextBox appear as though it has a lot of data in it, you would tell it that it does. As the user scrolls around, you would replace the data in the buffer with the relevant data from the underlying source (using random file access). So your UI would be monitoring a file, not listening for logging events.

  Of course, this is all a lot more work than simply using a StringBuilder, but I thought it worth mentioning just in case.

  HTH,
  Kent

  Steven Sudit : @Kent: If you look carefully at Paul Sasik's answer, including my comments, I think you'll find that it will not slow down with large strings. This is because we still use AppendText when writing to the TextBox and keep clearing the StringBuilder.

  Kent Boogaart : @Steven: but the `TextBox` still has all the data, thus you're still restricted in terms of how big your data can get. Virtualization does away with that since it only looks at a small window of the whole data set. You're only really limited by disk space.

  Steven Sudit : @Kent: Any cost/benefit analysis must include the costs. A virtualized textbox is much less useful, since you can't even do simple things such as Ctrl+A, Ctrl+C. Virtualization is appropriate in cases such as a grid attached to a database table.

  Kent Boogaart : @Steven: I'm simply pointing it out as an option. If you were to implement data virtualization, it's because you have large amounts of data. You *wouldn't want* Ctrl+A to work in that case because the user can shoot themselves in the foot. And even if you *did* want it to work (say, if there's less than 10MB data) you still could by virtualizing the selection. Or you could warn the user if they're about to copy a large amount of text (from the file) onto their clipboard.

  Steven Sudit : @Kent: Fair enough, data virtualization would be a correct general answer for how to handle unlimited amounts of data. I'll toss this an upvote so it gets noticed.