Saketa
by on October 9, 2020
38 views

Office 365 migration API leverages Azure to boost migration speed and reduce the throttling effect to facilitate heavy migrations without affecting the performance.

Office 365 Migration API

SharePoint migrations became much easier with the help of third-party tools available in the market. But when it comes to SharePoint online migrations, performance and throttling issues are always primary concerns no matter which tool we use. Because of this SharePoint throttling effect, sometimes SharePoint sites will also become unavailable and causes poor performance to the users accessing the sites.

To address these issues, Microsoft came up with separate API for migrations named O365 Migration API. This API leverages Azure and boosts the migration speed like ever before and the throttling effect will be greatly reduced, so users can use SharePoint sites without any performance issues even at the time of heavy migrations.

How Office 365 migration API works

To explain it in a simple way, you will export your content as a package and submit it to Azure. The timer jobs in Azure will pick your content and migrate them to SharePoint Online.

Let’s review the process:

 

  • Prepare a package (Content Package) of items/files that you wish to migrate and another package (Manifest Package) of XML files that define metadata, relations, and settings for your migration. If this sounds complex, do not worry We will soon see what they are and how easily we can generate them.
  • Once the packages are ready, simply push them to the Azure Storage account and start the migration job.
  • Before starting the job, a queue should be created and associated with your package.

This can be easily done using the available API commands. When the migration job gets started in Azure, this queue holds the Job progress information.

  • Timer Jobs in Azure will pick your content and push them to target SharePoint Online.
  • During this process, the migration job progress and status messages will be written to the queue.

You can either read the messages in the queue to track the success or failure of your migration or you can simply get the job status using the available API commands.

Once the migration is completed, you can delete your packages from Azure.

We just saw the migration process. Now we will see how this can be achieved using available API commands.

We can either use the CSOM version of Migration API and code in C# or use the PowerShell one.

To simply explain the process, I will show you how we can migrate the Library Items from SharePoint 2013 to SharePoint Online using SharePoint Online PowerShell cmdlets.

As of now, Office 365 Migration API does not support the migration of site/list schema. Only content like files, folders, and items can be migrated along with its metadata, unique permissions, and versions.

So, to migrate the content using this API, initially, a List/library with columns should be created at the target.

Steps to use PowerShell cmdlets to migrate from SharePoint 2013 to SharePoint Online

Before you begin

Step 1: Setup your working directory

Step 2: Export your SharePoint List/Library content as a Content Package

Step 3: Generate the Manifest Package

Step 4: Submit packages to Azure Storage Account

Step 5: Start your migration job

Step 6: Monitor your migration job

Step 1: Setup your working directory

  1. Set up your working directory by creating two empty folders in SharePoint 2013 Server.
  2. Create a Temporary package folder. (To store content package)
  3. Create a Final package folder. (To store manifest package)

Let’s name the Temporary package folder as a source-package and the Final package folder as a target-package. (you can use any names. This is just for reference.)

Note: To simply explain the process, I am breaking the code into multiple steps. So, make sure you run the code in the same PowerShell console otherwise you may end up with “Not Found” exceptions.

Step 2: Export your SharePoint List/Library content as a Content Package

  1. Export your SharePoint Library Content as a Content Package to your source-package folder in your working directory by using the Export-SPWeb cmdlet.
  2. Run SharePoint Online management Shell as admin and run the following commands substituting your values:

 

0365 migration api

Once the command executed successfully, a Content Package will be created at your working directory folder “source-package”. This package consists of .dat files (pointing to your actual files) and few XML files (which holds files metadata and other information related to Source)

Now let’s see how to generate a target Manifest Package based on this Content Package and target Site details.

Step 3: Generate the Manifest Package

The Manifest package is a package of XML files that define metadata, versioning information, permission information, user and groups, lookup relationships if any for your items and the settings for your import.

Refer to Content migration schemas for more details about these manifest files.

To simply generate the manifest package targeting your destination environment, run the following commands substituting your values:

office 365 api

You can find this cmdlet detail here ConvertTo-SPOMigrationTargetedPackage

Once the command executed successfully, you can see your Manifest Package at your provided target-package working directory location.

Step 4: Submit packages to Azure Storage Account

Now let’s submit the generated Content Package and Manifest Package to Azure Storage Account.

These packages will be created as containers in Azure Storage Account. So, provide the names you wish for them using below cmdlets.

Also, you must provide the Azure Storage Account name and Account key to push your content to Azure. You can get them from here.

api for 0365 migration

You can find this cmdlet detail here Set-SPOMigrationPackageAzureSource

Once the command executed successfully, you can see Content Package, manifest, and queue containers with your packages in your Storage Account.

Step 5: Start your migration job

Now let’s submit and start a migration job referenced to a previously uploaded package in Azure Storage Account.

You can find this cmdlet detail here Submit-SPOMigrationJob

Step 6: Monitor your migration job

At this step, Azure will take care of migrating your content to the destination. This process is a timer-job based, which means we have to wait for Azure bots to pick our package and start the migration to the destination.

You can view the job progress by viewing the real-time updates posted in the Azure storage account queue (the one you created in the previous steps).

microsoft azure

Azure will also log everything that happened in the log file, which it places in the manifest container in Azure Storage Account.

If there were any errors or warnings, .err and .wrn files are also created in the manifest container.

Or you can simply use the below command to track the migration job status.

You can find this cmdlet detail here Get-SPOMigrationJobStatus

Once the migration is completed, you can delete these containers from Azure Storage Account as they are just temporary storage for your migration process.

To import your content from file shares to SharePoint Online using this API, you can click here.

Limitations

The Site, List Schema cannot be created with this API. Only content can be migrated. So, to migrate them, you must depend on any third-party tools or custom coding.

A maximum package size restriction applies. So, if you want to migrate a large number of files with huge sizes, you must split them into multiple packages. As the packages increases, the complexity of the process increases. You have to take care of the number of migration jobs and azure containers created, awaiting algorithm to track the completion of each migration job, a proper retry algorithm in case of any job fails and the proper process for deleting the containers after the completion of each successful job.

Handling unresolved users (if users do not exist in the destination environment), lookup relations and few other things also should be taken into consideration when using this API.

Saketa Turbo Migration

The API addressed two major concerns of SharePoint Online migrations which are performance and throttling issues. Yet the API has many limitations as discussed earlier.

To overcome these limitations and yet not compromising the performance, Saketa built a powerful SharePoint migration tool that leverages this Office 365 migration API for content migration and can also create an entire Site or list schema.

Saketa SharePoint migrator tool also provides advanced mappings and configurations which address many migration concerns like orphan users, missing site templates, limiting versions and incremental migrations.

Few salient features of Saketa SharePoint Migrator:

  • Migrate from a vast set of data sources like file systems, network shares, Cloud storages (Box, Dropbox, Google Drive, One drive) and any version of SharePoint (2007, 2010, 2013, 2016, 2019 and O365 Online) to any version of SharePoint.
  • Import data from an Excel file to a SharePoint list with the ability to map the fields and order.
  • Filter data on any field and migrate the filtered content using CAML query designer.
  • Schedule your migrations to run in less busy hours.
  • Automation capabilities with advanced PowerShell cmdlets
  • Inventory, pre-migration and post-migration analysis and reporting.
  • Manage you entire SharePoint security from a single place.

To install and to get more details of advanced features of Saketa Migrator, check  the Saketa Migrator site.

Saketa SharePoint Migration Tool

Experience a high-speed, hassle-free migration with no data limits.

Posted in: Business
,  reacted this
1 person likes this.
,  reacted this