Rather than having stories and critiques sent out individually, I'd suggest you set up a website which only the organiser can access to add/change pages, though all those interested could access it to read (the address to be passed by PM/email only so it isn't available for just anyone to find eg here on a thread).
That would go some way to providing anonymity (though style over a 2000 word piece would almost certainly provide clues for regulars here) as eg if amw was organising, Hex would send her story to amw by mail, amw would put it up on the site, on a page of its own; anyone with access to read the site could critique/comment, but that critique would have to be sent to amw to appear (and she could hold it back if she thought it too negative/offensive). That way everyone can see both each story and all the comments, as in Critiques here, and therefore could comment on the critiques if need be. That might then avoid the problem of person A finding person B's story uncongenial and therefore critiquing harshly, when person C might love it and provide more/better help, and vice versa, since sometimes if someone loves a story he/she might not see flaws that someone less enamoured can pick out to improve it.
When you had everyone's stories all written, critiqued and amended, you could then set up a vote as to which should be included if you were all still keen on the idea of publishing at that stage.
As to websites, I know Gary offers free sites for Chrons members, but I don't know how that works regarding actually making them. I think Anya and Mouse have sites which are free for people to create their own sites from templates, and I know Moonfruit.com also has a free option though from memory I think it's restricted as to the number of pages you can have. (I'm pretty au fait with that site builder so could help in the setting up if needed -- it has some drawbacks but is pretty easy to use after you get used to it.)
Anyway, I'm watching the whole venture with interest, but I'm not throwing my hat into the ring quite yet, not least as I need to finish my fantasy, so even if I could squeeze the time to write a short, finding time to do my share of critiquing would be more difficult.