disabling sendmail from using dns
Using options such as FEATURE(`accept_unresolvable_domains') and FEATURE(`nocanonify')
but none worked. In fact I was generating so much traffic that sendmail would refuse to accept anymore commands. Thus I lost emails. Let me describe some background of the setup and the growth rate to explain the problem in more detail. Sendmail is configured to dump all mail to sendgrid, a Cloud provider that handles your email relationship for you. Thus there is no need to reverse dns or do any checks since Sendgrid does it for me. All I need is to dump the mail as quickly as possible to SendGrid.
I found that sendmail is not fast enough to dump mail to SendGrid. I could of used other transfer agents but I don't want to configure more things. The environment is very simple. PHP talks to localhost port 25 to dump the generated email to sendmail. Sendmail will try to connect to sendgrid, if so, block the client (Apache) until the mail is sent. The average send time is 1 second, thus the apache server blocks for 1 second per email sent. Blocking is bad especially with this case, since a social application could contact 1000s of users from a logged-in user's action. Let's imagine that one user generated event can contact 5000 users. If each email takes 1 second that takes 5000 seconds for that process to finish. Now that the problem is defined, lets find a good solution which can be handed off.
I picked Gearman. I now build the email and write to a gearman queue. Around 30 workers are listening to the queue and will dump the mail directly to smtp.sendgrid.net via the PHP module Swift. What was gained in doing this?
- Contacting 5000 email recipients takes 200ms since I am writing to memory
- Apache does not block for to long (220ms).
- I do not use localdisk since sendmail is bypassed. /var/spool/mail/mqueue is not used
- I save on diskio and cpu cycles
- I can now send at a higher rate with hardly any errors.
- I can scale with more workers