Mountain View 1600, CA 94043
+1 650-253-0000
name@example.com
Web
Site
Search
Register
Login
DNN Home
Services
-
DotNetNuke Support
-
CRM Systems
DNN
-
DotNetNuke Modules
-
Module Downloads
-
Module Upgrade Policy
DNN Support
-
Module Update News
-
Knowledge Base
-
License Management
-
Invoice History
Module Downloads
Blog
Store
Contact Us
-
About Us
Search
Home
1
2
3
4
5
Home
Support
Bulk Emailer
Send Rate WAY too slow
Previous
Next
7/10/2013 1:17 AM
Christopher
Joined: 2/17/2013
Posts: 5
Send Rate WAY too slow
(United States)
I love the BulkEmailer module features - easy to use and solid.
However, I just ran a job for 112,000 email sends and it took roughly 4.5 seconds per email to send (the DNN scheduled job took 230 seconds per batch of 50). Increasing the schedule frequency to run every 30 seconds doesn't help because DNN doesn't allow the same scheduled job to overlap (would probably cause dupe sends anyways).
I'm having trouble justifying to my client why this send will take almost 6 days to get out the door. These are corporate emails and we are whitelisted with the corporation - the last module I used was the Advanced Email Manager and it was about 1000% faster. That said, it was a piece of junk and not nearly as polished as your module.
Is there a trick I can use to increase the throughput? Is there some sort of hard-coded sleep between email sends that I can adjust? Any other ideas?
Please help, I'm stuck!
Caleb Blanton
President
Creativa Consulting, LLC.
7/10/2013 2:12 AM
DNN Module Support
Joined: 8/28/2006
Posts: 2065
Re: Send Rate WAY too slow
(Australia)
Thanks for the message, and I would agree that 6 days is too long for sending this type of email. Couple of things that you should be aware of though.
1. We get WAY faster results than you are getting. Something like 6-8 seconds to send 50, and while we are running fast servers, they are nothing super special and they are on a shared environment. So it is strange that you are getting results this poor.
2. Because of point 1 above, I would be checking your hosting environment first, as it may be possible to achieve a satisfactory result just by ensuring that you are on 1/2 resealable servers, with fast connections to your SMTP server. Slow SMTP connectivity will always slow things down. If necessary, put a SMTP local virtual server etc to locally pool the messages out if your connection is slow. That being said, get very good results with ASW SES service which is a slower service than local hosted SMTP. It still gives us a performance far superior to yours. Also SQL performance is important. (read the next point).
3. Because we are doing a bit in the back end. Token conversion, auto link tracking, read tracking, read online, auto unsubscribe linking, we are creating 1 email message that is unique per send. Each message at run time creates a job in the SQL table and the module iterates through the job table till done. This process does take more time with reading and writing to and from SQL, and it runs on the same application pool as your web server. We can't access that process to 100% process time, as this would make your website slow to a crawl while BE sends. So it is intentional that we are not going as hard and fast as possible.
4. Comparing to other systems is not so relevant, as it is apples with bananas. If we did not have a robust database driven process that can be interrupted and will auto resume, and non unique message content. i.e. simple email blast that relies on the good fortune of the app pool, we could send huge volumes as fast as your internet connection would allow. That's not what this module is doing. Likewise if we had written a dedicated online service that did not share resources with your website, we could blast as fast as your computer will handle. Again not what is going on here.
5. Another reason we slow things down a little to to avoid some of the problems that arise using your own SMTP services with regard to sender base spam blocking. If you just go as hard as you can for a short time, you really put your servers at risk of sender base blocking.
So I agree that your performance is not good enough. I suspect that something is going on with either:
Overall server performance
SQL performance
Slow or capped SMTP performance
Easy tests are:
1. Simple content - reduce links, and tokens to see if that helps.
2. Use Amazon SES to see if that helps with SMTP performance issues.
3. Find a way to allocate more resources like processor and memory resources to your existing site. (share app pools, or more virtual resources).
4. Try a known high speed server, like an AWS server to temp host a copy of the site and send through that.
We will discuss options at our end, but actually you are the first to mention this in a long time, and we have people who are sending over 1 million messages a campaign (yes that does take a few days).
7/10/2013 3:18 AM
Christopher
Joined: 2/17/2013
Posts: 5
Re: Send Rate WAY too slow
(United States)
I'm running on a huge dedicated server and the SMTP server is on the same server. That same SMTP server was what we sent with before and I know it can handle more throughput.
I'm pretty sure this isn't an issue with the server size or the SMTP performance.
One thing I realized I didn't mention (that I should have) was that I'm using your special Linked List feature for my distribution list. This is an SQL query that returns 112,000 records. That query takes a 1-3 seconds to run, but I would expect this would only be called once at the beginning of the batch job. If this SQL was executed for each email send, that would likely explain the performance issues.
Thoughts?
-C
7/10/2013 11:29 AM
DNN Module Support
Joined: 8/28/2006
Posts: 2065
Re: Send Rate WAY too slow
(Australia)
That's interesting.
Would be a good test if you could run a query to extract the data and put into a temp list. Then use that list for sending. It would be a good exercise to know what happens.
Although it sounds like you know just what you are doing. I have seen some dedicated servers sharing SQL servers, that run on very slow single hard disk setups. Don't suppose your SQL could be running on a non raid disk?
7/10/2013 11:32 AM
DNN Module Support
Joined: 8/28/2006
Posts: 2065
Re: Send Rate WAY too slow
(Australia)
Notice your icon is not loading in these forums. Did you actually load one up?
7/10/2013 2:52 PM
Christopher
Joined: 2/17/2013
Posts: 5
Re: Send Rate WAY too slow
(United States)
My SQL Server instance has a dedicated drive and this is a dedicated server that only hosts a handful of my clients - there is very little activity on the DB server. This box is big and can handle much heavier loads than just some email sends; it's got a Quad Core Xeon 5320 and 8GB of FB-DIMM RAM.
We'll try to upload our list into a standard list and see if that helps performance. We'll also run an SQL Server Profiler on the server during the run to see if we can spot any unusual database utilization during the email send.
The next send will be a couple of days - during this time, is there any way you could do a static code analysis to determine if the queries associated with Linked Lists might be executed for each send? Seems like any easy thing to spot in code.
Thanks for the great responsiveness - it's nice to see your support is as great as your module!
P.S. I updated my photo in my profile. I saw the broken image in the Forums as well but when I went to my profile, the image appeared. Re-uploading the image seemed to fix that.
-C
7/11/2013 6:59 AM
Christopher
Joined: 2/17/2013
Posts: 5
Re: Send Rate WAY too slow
(United States)
Update:
We just kicked off another process and instead of using a Linked List, we ran the same script to use the Import List feature. This time, the number of emails was about 20,800 and each scheduled run takes 40 seconds or so. What's interesting to me is that the size of the email list does seem to impact the send rate for each scheduled process run. There seems to be a consistent ratio - when we sent 110k emails, the job took 220-250 seconds to run. With 20k emails, it takes 38-45 seconds to run.
Here's some more information on how we have the Bulk Emailer installed/set up:
DNN Version: 7.0.3
Bulk Emailer version: 62.6.7
Key Bulk Emailer Settings:
Authenticated users only: False
Server Mode: Host Settings
SMTP Server Mode: Rotary
Email Tracking: False
Disable Base URL: False
Email List Match: False
Enable My Tokens: False
Enable NVelocity Tokens: False
Email Tracking, Unsubscribe and View Online URLs are all left at defaults
Load jQuery: False
Server Hosted on WebFarm: False
CutyCapt: Not Installed
Amazon WS SDK: Not Installed
The size of the email list shouldn't affect each job run, but it certainly appears to have a correlating impact on performance. For the next big batch we send, what I plan to do is revert back to a Linked List and set up my query to only return the TOP 100 rows. If my theory is correct, this will change the job run time toa few seconds (assuming there is a base overhead with SMTP server and DB server interaction and assuming what I've observed that 0.0018 seconds per email in the list is added to the job run time).
I will report back with the performance after changing back to a Linked List with a TOP 100 added to it.
-C
7/11/2013 6:21 PM
DNN Module Support
Joined: 8/28/2006
Posts: 2065
Re: Send Rate WAY too slow
(Australia)
Thanks for the information. This is very interesting.
I assume that the mail message and number of links etc are the same?
7/11/2013 7:10 PM
Christopher
Joined: 2/17/2013
Posts: 5
Re: Send Rate WAY too slow
(United States)
I am using the exact same content in both cases.
-C
7/11/2013 7:38 PM
DNN Module Support
Joined: 8/28/2006
Posts: 2065
Re: Send Rate WAY too slow
(Australia)
Thanks.
Page 1 of 1
Previous
Next
Home
Support
Bulk Emailer
Send Rate WAY too slow