February 19th – Management Reports are now turned back on and be actively used at your site(s).

We wanted to provide an update on Management Reports. They remain turned off as we continue our investigations related to the issues experienced on Tuesday, February 12th. However, today, you will notice that you can now access the Management Reports menu, select a report, choose a date range and EFIN and click to run the report. When clicking to run the report, you will see the below screen. This is expected behavior at this time since Management Reports are turned off.

Update 8:46 am EST:  All display issues have been resolved.  You may need to do a hard refresh (Ctrl+F5) if you are still experiencing any display issues.

Update 8:30 am EST:  2017 and 2016 are now displaying correctly in all browsers.  We are currently looking at 2015

The display when editing a return is not behaving as expected.  We are in the process of correcting this.  This impacts 2017, 2016 and 2015

Issure ID 2480 is now resolved. For impacted returns, open each return and go to the Filing Status screen within the Federal Basic Information menu. Make sure “Married Filing Separate” is marked then clicked Continue and follow the prompts. You will be presented with the question to indicate if the TP and SP lived together at any point during the year.

Issue ID 2480: MFS Question Removed to indicate if SP lived with TP at any point during the year

• Can impact IRA Deduction and SSB calculations

(COMPLETE) Update 4:25pm EST: All deployments have been completed. At this time, loading the MA state program should function as expected. We apologize for the inconvenience this has caused your sites, volunteers, and taxpayers.

Update 4:12pm EST: We are now deploying to the last set of servers.

Update 4:04pm EST: We have finished deployment to 24 servers and are now deploying to the fourth set at this time.

Update 3:53pm EST: We have finished deployment to 16 servers and are now deploying to the third set at this time.

Update 3:41pm EST: We have finished deployment to the first 8 servers and are deploying to the second set at this time.

Update 3:30pm EST: We are in the processing of deploying to the first set of 8 servers.

Update 3:22pm EST: Our current level of concurrent users on the site is now at an optimal volume to proceed with the second rolling deploy to the web services to minimize potential disruptions across all sites. This should take approximately 1.5 hours to complete and will begin at 3:30pm EST.

Update 12:36 pm EST:  Due to the number of concurrent users on the site, we deployed to the calculation servers first in an effort to minimize potential disruptions across all sites.  We are currently watching the number of concurrent users in order to conduct a second rolling deploy to the web services.

For those sites impacted by not being able to add a MA return, please make sure you utilize the return tag feature so you can go ahead and prepare the federal return.  You can also transmit the federal return and then use the State Only feature to send the state once you can add it to the return.

Update 11:56 am EST:  We are deploying to the last set of servers

Update 11:11 am EST:  We have deployed the fix to 24 services and are deploying the next set of 8 now (and yes, it is am in GA so I updated my other times

Update 10:59 am EST:  We have deployed the fix to 16 servers and are deploying to a third set of servers now.

Update 10:41 am EST:  We have deployed the fix to 8 servers and are deploying to a second set of servers

Update 10:26 am EST:  We are in the process of pushing out a fix to the MA loading issue.  We are doing this slowly across a few servers at a time to minimize potential disruptions.

We are aware that there is problem loading  MA returns.  The system will not let you add it to an existing return.  If you have a return with MA on it, you may experience a long running calculation.  If you can select the left navigation bar, select the personal information section to break the calculation.  You may want to delete the state and tag the return that it still needs MA to be added when available.

All areas of the program are operational except Web Reports.  We will continue to optimize how the reports run with a goal of getting them turned on my Monday, February 18th.

Update 2:03 am EST:  All areas are operational except 2015 and web reports.  You can print existing 2015 returns, but cannot open the return or start a new return

Update 12:57 am EST:  Printing for all years is operational

Update 12:42 am EST:  2018 printing is now operational from outside and inside the return.  We are still working to resolve prior year printing issues

Update Midnight:  2018 printing:  You can print from summary/print.  If you get the message that the PDF failed to load, click the reload button and it should render a PDF.  We are continuing to work on printing from the Client List and the Submission page in e-file

Update 11:44 pm :  We are working on bringing printing back online

We will be performing maintenance on the site from 11:00 pm EST to Midnight.  During this time, you may experience some slowness and/or small pockets of outages.  We will be updating the blog post with any anomalies we see during this time.

UPDATE 2/13/19 – Impacted VA returns can be resubmitted at this time.

Scenario:  My VA return is rejected for one of the following codes:

• 760ADJ-020
• 763ADJ-073
• 760PYADJ-140

Solution:  We will be making a code change.

Action: Please note that some impacted VA returns have been resent and acknowledgements will come back. This is due to being able to identify VITA specific returns internally and manually fixing. Check impacted returns over the next 24-48 hours to see if accepted acknowledgements have been returned.  If returns need to be resent, we will update this post to let you know when the fix is live and the impacted returns can be resubmitted.

Update 5:41 pm:  We are turning real time calculations (refund monitor and AGI monitor) back on

Update 5:30 pm:  We are bringing the rest of the servers online as we are seeing response times in the normal operational range.  We are continuing to monitor the changes to the system as we bring everything back online.

Update 5:04 pm:  for those sites coming on board and those who are following the blog post, we are continuing to work on the errors and slow response times.  We currently cannot give you a resolution time.

Update 4:46 pm:  We have duplicated the errors you are seeing in the application and are working to determine the cause of the errors and slowness.

Update 4:24 pm:  The sytems are still booting up so you will experience slowness until all systems get online.  You may also receive a an unexpected error because of the lengthy wait time

Update 4:14 pm:  We are preparing to bring the first 18 servers out of standby

Update 3:45 pm: We will soon be placing the site in maintenance for a period of 30-45 minutes to allow our engineers to perform more granular investigation. We apologize for the inconvenience this continues to cause.

Update 2:37 pm:  We are turning off “Real time” calculation (Refund monitor) to do some analysis

Update 2:34pm:  We are seeing some lag time internally that we are currently investigating.

Update 2:13 pm:  The web services have completed.  We are seeing significantly less lag time going between pages.  We are continuing to monitor.

Update 1:46 pm:  We are doing a rolling reboot of our webservices that take the connections between you and our other services.  We will update the blog when this process has been completed.

Update 1:42 pm:  If you stuck opening a return, click on the Personal Information Section in the left navigation bar (if it is visible to you).  This will break the cycle and allow you to work through entering data.  The return will be slow calculating through pages.

Update 1:31 pm:  We are continuing to determine what is causing the slowness between pages.  

Update 1:18 pm:  We know it is taking a significantly slower than usual time for some users to open existing returns and move between entering the SSN and the personal information page.  We are currently looking to see the cause and find a solution.

1We are receiving information from sites that there is a sudden slowdown starting returns and going between pages.  We are currently looking into this.

12:28pm ET

The deployment has been applied to all servers. At this time, impacted sites with Rhode Island returns should now be able to access Summary/Print, see the Refund Generator with Results, and access E-file. Please let us know if you experience additional issues.

11:30am ET

The deployment for the fix is still occurring. As noted in the 10:47am ET update, deployment across all servers will take an hour or two before complete.

10:47am ET

The fix is currently being deployed at this time. For impacted sites with Rhode Island returns, calculations inside the returns should start to appear as the deployment continues. This will continue over the next couple of hours as the deployment continues across all servers.

10:37am ET

We anticipate having this resolved soon.

10:17am ET

Testing is still occurring in our pre-production levels before deploying live.

10:01am ET

The fix is currently being tested in our pre-production levels.

9:49am ET

Our engineers believe the issue has been identified and are working to resolve.

9:35am ET

Issue 2699 has been opened to investigate sporadic reports of the calc breaking in a return when a Rhode Island state return is added. We are looking into this.