Writer's Nightmare: Not backing up

[Ray McCarthy]

There are USB drives whose casing is made of metal. There are no parts in them that can be broken by an average man.

The connector will still break as easily, but a plug cut from a cable can be soldered on. They are not really more robust.
The SD cards and USB sticks (unlike real HDD) fade even when powered down in the drawer.

 

[Vertigo]

Don't rely on any single mingle method is important too. Webmail is easy to lose via account stealing, cloud services can be hacked ( what would happen if one suddenly went out of business?) physical drives such as usb drives can suddenly jus drop dead ( ive had a few thumb drives die on me) .
So, i use dropbox, save back ups localy on my notebook and use a usb drive. I guess its better to be paranoid than sorry.

Never mind hacking cloud drives, a simple Google 'bot seems to be able to do it. I did a google search for a book title a while back - just looking for info on the book - and when I clicked one of the links I found myself looing at the complete text of the book. A little research showed that it was within a cloud service account. I notified the author who was not amused; it turned out his publishers were using the cloud service for - you guessed it - backups!

 

[Ray McCarthy]

Loads of Cloud services and Social media simply rely on link not being public, no actual security at all.

 

[Vertigo]

That appears to have been exactly the case in this instance.

 

[Wanderlog]

The connector will still break as easily

Nope. There are models with no protruding connector at all. Like Kingston DataTraveler SE9 G2 3.0. You have to be the Superman to break it with your fingers. I started using one of such drives after I lost a couple of other similar devices due to broken chains attaching them to my key ring.

when I clicked one of the links I found myself looing at the complete text of the book. A little research showed that it was within a cloud service account. I notified the author who was not amused; it turned out his publishers were using the cloud service for - you guessed it - backups!

This is a typical error made by users who have little knowledge of basic IT concepts. In particular, people often put their files on the local network file shares or even on the Internet without properly assigning access rights and, in fact, making their data available for everyone. However, accessing such data is not "hacking cloud drives". It's rather stumbling on it.

 

[Vertigo]

That's why I said "never mind hacking." But I would say it is rather more than just stumbling, since the search engines have managed to find it and I don't think the search bots are in the habit of just 'stumbling,' they follow very specific algorithms. And in this case, according to the author, the publishers were simply using a commercial cloud service and had no direct control over file access themselves. they just trusted the cloud service (which was probably very stupid).

 

[Wanderlog]

I don't think the search bots are in the habit of just 'stumbling,' they follow very specific algorithms

In general, they are simply following all links they find on different web pages. If they find a link to a cloud drive and can follow it, it means the access to the content wasn't properly secured. Search bots don't implement any algorithms allowing them to find hidden and protected content. Moreover, there are simple means to prevent them scanning a specific web site.

There is one very unpleasant circumstance, though. First, if you use Google Drive for content sharing purposes, you can either give someone passport-protected access to it or just generate a web link allowing anyone to access it. The first method requires the user to have a Google account (and not all people have it), so the second one is often used. It's OK if you don't put this link on a publicly accessible web site. However, here comes the second issue. Google automatically scans mailboxes of its users for different purposes. For example, it you book your hotel on the Booking.com web site and it sends a confirmation to your Google mailbox, you can open the web mail interface and see that the confirmation letter has special buttons attached to it. In addition, the location of the hotel will be automatically shown on Google maps of the area you open. So combining those two issues, you can have web links from your messages leaked to Google bots. To its search bot, in the first place. While it might be done unintentionally, it still has grave consequences.

I suspect that the publisher simply shared a folder on its Google Drive for everyone and then sent a link to someone owning a Google mailbox. The link leaked to the search bot, and voila! The content can be accessed by anyone on the Internet.

The moral of the story is, always secure others' access to your cloud data with a password and never keep unencrypted data there even if you give the access to no one. You never know who might become interested in your data, a Google's system administrator or a CIA employee, whoever...

 

[Ray McCarthy]

Moreover, there are simple means to prevent them scanning a specific web site.

Lots of bots ignore robots.txt actually hackers look at it in case someone thinks it stops bots to get a list of "interesting" links.
If there are link(s) on a site(s) already indexed those pages are public.

Some links follow certain rules, this was problem with dropbox in the past. The links could be "guessed". Also Twitter and Facebook often have links in pages that a bot can "see" but are not visible. The bot is just parsing the source file.

The Search engines use "who is" databases too, to find new domains. If that has a robots.txt file in the root, you just told everyone a lot of the pages.

always secure others' access to your cloud data with a password and never keep unencrypted data there even if you give the access to no one.

The problem is that the Cloudy company won't tell you how their security works and the password might be useless. Only proper encryption works.

 

[Wanderlog]

The problem is that the Cloudy company won't tell you how their security works and the password might be useless.

Well, in this particular case (Google) we know how it works. Google secures access with its accounts. MS secures access to its OneDrive (or whatever it's called now) using Live ID credentials.

However, we know that MS, for another example, scans the content stored on OneDrive. I recently read about a pedophile who was convicted on the basis of pictures found on his OneDrive by the MS search bot. While keeping such pictures is a crime, I hate the very idea of a Big Brother watching me permanently (for my own safety, for instance).

So I can't but agree with you. Encrypt, encrypt and encrypt again. Or, even better, make an archive and then encrypt it.

 

[Ray McCarthy]

Well, in this particular case (Google) we know how it works. Google secures access with its accounts. MS secures access to its OneDrive (or whatever it's called now) using Live ID credentials.

No, that's just the interface the regular user sees. We don't at all know how it's secured, what aliases to same storage there might be. Or anything.

Google and MS both scan storage. Google scans email too. There is/was a class action about Yahoo's treatment of email.

 

[Ray McCarthy]

Or, even better

Don't use them for anything important.

 

[Vertigo]

Wouldn't dream of it

I keep all my stuff backed up to a rotating set of three hard drives plus another two offsite. Me paranoid?

For our lab where we have slightly larger data requirements we do the same but this time it is 16TB NAS drives that we use.

 

[Wanderlog]

We don't at all know how it's secured, what aliases to same storage there might be. Or anything.

True. But we can say the same about virtually every service we don't control directly. I fact, I can guarantee the safety of only those local file systems in classic OSes I installed and tuned myself. Everything else is a dangerous area from my point of view. However, it doesn't mean that we should never believe cloud storage providers.

Don't use them for anything important.

A correction: don't use them as the ONLY CARRIER of anything important.

There is an axiom: the cost of a backup system shouldn't exceed the cost of data it protects. The cost of data a writer generates can be calculated, for example, on the basis of their per-hour payment. Or per-word payment multiplied by the number of words they can generate per hour. Usually it's not much, so a mix of flash drives and free cloud storage should be sufficient in most cases.

Me paranoid?

Not yet, although three different HDDs are already close to an overkill. You should have two or three copies of your data, and at least one of them should be an offline copy (i.e. a detached HDD or a flash drive). BTW, shadow copies of you working directories can be a way to restore old version of files. However, they don't protect the data from hardware failures and logical corruption of the file system.

For our lab where we have slightly larger data requirements

That's different. There is a number of tools available in a corporate network environment that are inaccessible for home users.

 

[Dennis E. Taylor]

There is an axiom: the cost of a backup system shouldn't exceed the cost of data it protects. The cost of data a writer generates can be calculated, for example, on the basis of their per-hour payment. Or per-word payment multiplied by the number of words they can generate per hour. Usually it's not much, so a mix of flash drives and free cloud storage should be sufficient in most cases.

This calculation should include the replacement cost of the computer equipment and furniture that will be destroyed when the writer discovers that they've lost their WIP.

 

[Ray McCarthy]

The cost of a backup system shouldn't exceed the cost of data it protects

It's not as simple as that. How do you calculate the "cost" of the data and replace it if you have that amount of money?
The data may not be worth much yet, but irreplaceable.
It may be required for legal reasons.

 

[J D Foster]

I feel the pain! I had a short story I finished then I accidentally saved over it with an earlier version. I lost 20k words. I spent days trying to find an older version of it with more words. I got all these recovery tools off the Internet and tried them out on every computer or hard drive or SD card I had that I thought might have a back up of the story on it. It was all for naught though I couldn't find a more complete version... Cue weeks of wallowing in despair and refusing to work on that particular story.

In the end I re-wrote it, maybe 6 months later and I was surprised how much of the original story I remembered. In addition I think the new version is better and I added an extra 10k words to the word count.

I keep all my stories backed up on Google drive now which has pretty good recovery functions, and keep a copy on an old external hard drive. The whole thing taught me to 1, be more careful and 2, don't fear loosing words as they can always be re-written.

Having said that it was only 20k words I can't imagine what it would be like if one lost say 80k or more words....

 

[Warren_Paul]

This thread reminds me of Memories of Ice by Steven Erikson -- the third book in his series. Originally it was supposed to be the second book, following Gardens of the Moon, but he suffered a hard drive failure and lost all his work on it. (about 300 pages).

He ended up writing The Deadhouse Gates instead, which became the second book. Afterwards he went back to start again with Memories of Ice. He rewrote it from scratch, and to this day it is considered the best book in the series. A lot of people believe the fact that he had to start over completely from scratch is what made it so good.

So I guess good things can come out of the bad?

 

[millymollymo]

Ratsy, I hope you have salvaged this, I hope the frustration and heart sinking anger has also moved on.


I have a corrupted USB - it converts everything to a language that appears to be of Asian origin. I am also the not so proud owner of a dodgy SD card port, that sometimes will read and others meh.

(Thinking of which, have you attempted to open the file on any other machine, or copy saved to another drive? Bizarrely, this has worked for me in the past. )

Even so, I still don't use cloud save features for my work.
I save to the main drive. I copy the save to a secondary USB. Once in a-when-I-remember-to I shift that across to another (desktop) machine.
One day when I am rich, I'll buy a portable HD, right now this messing about works for me.

 

[Wanderlog]

This calculation should include the replacement cost of the computer equipment and furniture that will be destroyed when the writer discovers that they've lost their WIP.

A good point.

How do you calculate the "cost" of the data and replace it if you have that amount of money?

I described the basic method of calculation based on work time spent to create such data, but it really depends on the writer. Some people (like me) hate to lose even a single page, and others, recalling the discussion in the other thread, routinely delete and rewrite big parts of their texts. My point is, no need to be over-zealous in this matter.

It may be required for legal reasons.

It's for corporations, not for home users. Enterprise-level storage and mail systems have built-in retention tools allowing them to keep non-destructible copies of data. They are great, but unfortunately, such systems cost tens of thousands.

 

[Ray McCarthy]

It's for corporations, not for home users. Enterprise-level storage and mail systems have built-in retention tools allowing them to keep non-destructible copies of data. They are great, but unfortunately, such systems cost tens of thousands.

With moving to online etc (non-business) and home businesses, yes there are legal needs to backup. You can't trust businesses and Gov departments not to loose stuff either.

No, Backup is separate and needs properly managed, not included in every "corporate" purchase. What about smaller businesses?
Tools prices starts at "free" (open source).
Some systems can be expensive.

There is a lot more than creative writing. Even for that if you consider many years it may not be possible to re-write it.
Or retake the photos, video etc.

There are lots of one man show home businesses. Everything from A to Z. many have no more time, money or expertise than a hobby writer. There are many kinds of full time writing career too, not just novels or fiction.