Please be aware we are experiencing issues resulting from the weekend rollout of the Media Library release. You may notice the following issues:
- Missing images on blocks, or labels for blocks appearing and overlaying page text
- Photo gallery navigation numbered lists showing up
- Your submenus and social media blocks have been completely reset
- Priority link/vertical focal link images popping out of their frames
Update 7
Mon, 14 Dec 2020 - 1:27 PM
The following updates were made over the weekend to address on-going issues related to the Media Library rollout still affecting many sites.
Person content type - Documents attachment section had been inserted in the wrong location in the content order. The Documents section has now been relocated to the proper spot at the bottom of the page.
Google Analytics - During the media update we also included the newest version of the analytics module. It became apparent after the rollout is that it no longer supported how SiteFarm uses it. As a result, your analytics are missing a week's worth of data spanning from Sunday, December 6 through to Sunday yesterday around 2:30am. If you use Google Tag Manager, this data should not be affected.
Twig Template update - we've revised our twig templates for teasers and other view modes to support not only the new media field names, but also the original image field name, which will assist with some custom work to continue displaying images that were present prior to the media release.
Photo gallery captions - The caption text now displays in the correct place and draws from the proper caption field to populate it.
Addition of new, temporary module called "Media Fix" - This module is intended to assist you in determining if any of your images are missing from default SiteFarm sections of your site (pages, blocks, photo galleries, and categories).
The module is available, but disabled by default. If you suspect that your content is missing images, a user with the site builder role can enable it themselves, or else a site manager can email the SiteFarm team to enable it for them.
Once enabled you can locate the service in Shortcuts > All content where you'll find four new tabs:
- Fix images
- Fix categories
- Fix galleries
- Fix blocks
For images, categories, and blocks:
- Check to see if any images are listed as missing
- Check to see if the image in question is actually in your media library, if so go to the page and attach the image
- If it's not in your media library, right click on the "Old primary image" image and select "Save link as..." or "Open link as..." so you can download the full-sized image, not "Save image as..." for the thumbnail as you see it on the screen or else you only have the tiny version of your picture.
- Edit your page and attach your downloaded image, which will then be added to your media library as part of the process.
For galleries:
- Each gallery page offers a one-to-one comparison; the left column of images should match the right column next to it.
- If an image doesn't show its match, this indicates a missing image. Repeat steps 3 and 4 as described above to replace it.
- Refresh the galleries tab list and continue checking your list.
IF, across all four tabs, it appears you have more than 40 entries, you are welcome to contact the team at sitefarm@ucdavis.edu to rerun the script to fix this, though be aware any new content you've added will need to have its media reattached, which will be available from the library. If you have 40 items or less, or you've added a lot of new content, we recommend undertaking this task manually yourself as the length of time involved for us to fix it is likely greater than the hour of work or less you might need to make the adjustments.
We recommend Site Builders uninstall the module when your review is complete, or you're welcome to contact us if you need assistance in doing so. The module will be removed from the service at a later date, but uninstalling it helps save system resources on your site for better performance.
We appreciate your patience and assistance as we reviewed and implemented solutions. Should you spot any issues in any of our default offerings please be sure to let us know.
Update 6
Wed, 9 Dec 2020 - 8:46 AM
The patch to fix the issues related to the Media Library rollout is now complete. If any issues persist, please let us know as soon as possible.
Once again, a big thank you to everyone for your patience and assistance.
Jessica
Update 5
Tues, 8 Dec 2020 - 1:43 PM
Hi everyone,
We're about halfway through the patch update at this point and we're expecting to finish up by about this time tomorrow, if not sooner.
The focus of this update is the Client error: `POST https://api.fastly.com/service//purge` resulted in a `404 Not Found` response: {"msg":"Record not found", "detail": "Cannot find service '(null)'"} error.
We've been clearing it as you've been reporting them, but as soon as you edit or add new content to your site, your site sends a call through the API to the Fastly servers we use and it triggers the error message again.
If you need to make edits during this patch, then please note the error message will return but be cleared when your site is updated with the fix patch. If you can avoid making updates, the error should not reoccur. In either case it should be fixed with this update so unless it's urgent, we're asking you to put a pause on submitting requests for the error to be removed. Any tickets you submitted via our sitefarm@ucdavis.edu support venue will be closed in anticipation of the patch addressing this issue.
I've also want to clarify that this patch contains fixes for:
- media outage
- Fastly error message displaying
- >>>>>dev text artefact in person profile pages
- priority link images popping out of frame
What's not included:
- sub menus and social media blocks
These two blocks have been permanently reset to their original state. This was not intended to happen and we're taking steps to make sure this doesn't happen again in the future. You will need to review your submenu and social media blocks to either edit and tweak them and possibly their position on your pages or remove them entirely if you weren't using them in the first place.
Thanks,
Jessica
Update 4
Tues, 8 Dec 2020 - 9:18 AM
Good morning everyone,
The fix tested on our TEST servers late yesterday was a success. Shawn reports the rollout to fix our LIVE sites began approximately 9:00 PM last night and, due to the large volume of sites on the platform, has completed about 460 sites so far with 760 sites remaining. Since the process began 12 hours ago, you can see this will take a while.
Some of you may notice your site is already fixed if it was part of the initial 460. If you had to add any media to pages or blocks as part of any critical fixes on major pages, please check those today to see if the media is still present or if you need to edit the page/block to reattach your media from the media library. If your site is still to be updated with the fix, we continue to recommend avoiding adding media unless it’s related to critical areas of your site due to the need to reattach them after the update occurs.
Also, if you see an error message on your site that looks like this:
Client error: `POST https://api.fastly.com/service//purge` resulted in a `404 Not Found` response: {"msg":"Record not found", "detail": "Cannot find service '(null)'"}
Please let us know. I would like to emphasize that nothing is wrong with your site (it's a backend message related to our Fastly service) but we need to manually clear the message from each individual site where it appears and it can only be done by a member of the SiteFarm team.
Again, we thank you for your patience and we hope to have everyone's site back in working order very soon.
Thanks,
Jessica
Update 3
Mon, 7 Dec 2020 - 4:36 PM
Great news! We've found a fix!
What's happening next
We're going to put the fix on our TEST servers first to make sure everything works the way we think it will to resolve the issue with your images. I don't have an ETA yet, but I should be able to let you know sometime tomorrow if it a) worked and, if so, b) when we'll then roll out the fix to everyone's LIVE site(s).
What should YOU do next?
Because of the nature of the fix we're asking you to do everything you can to avoid adding new media (images, files, etc.) until the fix is added to your LIVE site. That said, if you have critical updates you have to make that need media, please do so but make a note somewhere of the page/block and the image or file you've associated with it. New additions attached to pages or blocks will be wiped out when we roll out our fix, but the media you add related to those pages/block will still be in your Media Library; you'll just need to reattach them. This advisement is to help you avoid having to do too much manual fixing after we release the patch.
A Thank You
I added this in the slack channel, but it bears repeating: I really want to thank everyone for your patience today. This was a significant blow-up with a large impact on all your sites and we recognize how vital it is that your sites remain up and functioning properly. It’s hard to express our appreciation for the fact all this happened and, instead of being rightfully frustrated and angry, all of you jumped in and participated in helping us track down issues so we could work on them. We’re really lucky to have such a wonderful community of users—it’s a pleasure to work with and for you.
Thank you,
Jessica
Update 2
Mon, 7 Dec 2020 - 1:53 PM
While the big issue of the missing media is on-going, we now have hotfixes queued up to fix:
- the ‘>>>>>dev’ artefact text on the person profile pages,
- the photo gallery navigation buttons styling
- vertical focal link /priority links popping out of their frames
I’m waiting to find out when we can release those, if it will be combined with the missing media fix or if we can release those more quickly.
As ever, thank you for your continued patience while we work to resolve the problems.
Jessica
Update 1
Mon, 7 Dec 2020 - 12:31 PM
The team looked at where we’re at this moment and decided rolling everything back is a bad idea since it could mean the loss of data that may have been added on your site during and after the rollout, especially as it relates to form submissions.
Instead, we’re going to write fixes to automatically reinsert your images back into the blocks. The difficulty at the moment is that I don’t have an ETA for you because the team needs to fully identify the affected code and write new scripts to account for the missing images.
What you can do right now:
- Fix any blocks you are especially concerned about right this moment, like on your home page or on significant landing pages in your site. All your images should be in your Media Library, it’s only a matter of searching through the library to find the image you need and re-inserting it.
- Sub menu and Social Media blocks in the sidebar: these were completely reset during the migration. This should not have happened, but did so because these two blocks were included in a reversion process when they shouldn’t have been. What this means is you are welcome to delete or edit these two blocks right away. If you had the social media block set up with your own accounts, you will need to redo this block.
We’re also looking at the following issues:
- Blocks: as mentioned before we believe we’ve identified the underlying cause contributing to the disappearance of your block images, related to the migration process. I’m still waiting on the specifics of the failure so I can offer a more detailed explanation. We recognize that some of your sites are quite large and we want to help you avoid doing any more manual work than strictly necessary to address critical public-facing pages.
- Photo galleries: a theming issue combined with third-party javascript change is resulting in the weird formatting you might be seeing with your image galleries
- If I’ve missed anything that doesn’t fall in the three categories (block images, social media/submenu reset, photo galleries), I’ll be re-reviewing all your tickets and posts here in the #sitefarm channel to make sure we don’t miss anything. Please note that restoring your images is our first priority.
Initial Announcement - Media Library Rollout Issues
Mon, 7 Dec 2020 - 10:08 AM
Hello everyone,
If you’ve visited your site(s) this morning you have likely discovered several issues resulting from our Media Library rollout over the weekend. If you haven’t, then this is an advisement of the rollout issue.
According to reports so far, these issues may include:
- The complete reset of submenus and the social media block
- The presence of >>>>>>dev text in your people profiles
- Missing images in blocks, most notably marketing highlight and marketing highlight horizontal, but other blocks have been mentioned too
- Missing images in the photo galleries
The team is currently investigating these issues. In the meanwhile, we’ve been inviting our users to keep an eye on the Slack #sitefarm channel where we’ve been taking reports of issues and asking users to look/try/report on specific problems and generally joining the discussion.
Our apologies to you that the new additions to our SiteFarm service didn’t go as planned. Given how excited we were about this release, it’s a bummer to see the problems we’re having. We appreciate your patience while we work on it and try to find fixes as soon as possible.
If you haven’t joined our #sitefarm Slack channel yet, here are instructions for doing so:
Join the #sitefarm channel
- Get the Slack application if you don’t already have it. Check with your IT support if you need assistance with this step.
- Click the UC Davis Aggie workspace button.
- Click on the Channel title
- Use the search field for ‘sitefarm’ and select it from the list.
- Click the Join button located at the bottom of your screen. The #sitefarm channel should now appear under your Channels list.
Thanks,
Jessica