Re: Fastest way to inject a lot of mail?

John Levine <johnl@taugh.com> wrote:

According to Claus A.mann  <INVALID_NO_CC_REMOVE_IF_YOU_DO_NOT_POST_ml+sendmail(-no-copies-please)@esmtp.org>:

John Levine  wrote:
>

 Is there a faster way to do it? SMTP to 127.0.0.1? LMTP?

>
First you need to identify the bottleneck(s),
then you can work on solutions.

>
Well, yeah, that's why I was wondering whether running the sendmail program
is likely to be slow.
>

BTW: did you read the fine documentation?
(hint: "TUNING"...)

>
I did and unless I missed something, it says nothing about injecting
mail via the sendmail command other than the obvious thing that you
want to queue rather than delivering synchronously.
>
So here's a question: I have on the order of 10,000 messages, each
with a dozen or so recipients. It's currently running the sendmail
command for each one. If I opened a connection to 127.0.0.1 and
did a sequence of MAIL FROM/RCPT TO/DATA, would that be faster? How
about if I did it with N processes in parallel for some modest N? It
currently takes about 6 hours on a moderately fast VPS.
>
If nobody has any idea, OK, but it's hard to believe I'm the first person
ever to wonder about this.

Hints about injecting a few thousand emails to local sendmail:
1. Inject multiple messages via the same SMTP connection
   to avoid needles forking of local sendmail processes
2. Group recipients by recipient's domain
   to reduce number of outgoing SMTP sessions
3. Use parallel SMTP injections with VERB SMTP command
   to avoid merely queuing due to high local system load.
   VERB turns on "sequential delivery" with reporting delivery progress
   so your parallel submits won't merely put messages to the queue for
   later delivery.  It may require allowing VERB from 127.0.0.1 *ONLY*.
   Start with a few SMTP sessions. Consider slow increase to a few
   dozens.
AFAIR Sympa mail list manager provides some useful hints.
Queue fast and send out fast does not mean the same for mass mailing.
Anyway:  Expect a few+ surprises from anti-spam measures of receiving
servers, fresh/new surprises *too* .  Usenet message short advises
must be incomplete so do expect a few nasty surprises.
About lack of recipes: People tend to avoid providing too easy recipes
also for (incompetent) spammers.
--
[Andrew] Andrzej A. Filip