Limits with google drive auto update

More
4 years 9 months ago #505 by intacto
Hi,
First of all thanks for the wonderful plugin!
We have started using your plugin on a recent game we released and have a small concern with the auto update system.
Are there any rate limits on google drive auto update? Since the game we released will get millions of downloads after the planned marketing campaign we are curious about the restrictions google may have put on the system i2 locaization uses.
Could you please shed some light on this issue. Also whats the correct way to use our own server instead of google's for this purpose.

Thanks,

Cumhur

Please Log in or Create an account to join the conversation.

More
4 years 9 months ago - 4 years 9 months ago #508 by Frank
Hi,
I'm really glad you find the plugin useful and congrats on your game!

we are curious about the restrictions google may have put on the system i2 locaization uses


By looking at the Google Script Daily Limit , the checking I'm doing to get the spreadsheet last modified date, doesn't have any limit. Unless it is considered a URLFetch (which as far as I understood are for fetches initiated within the webApp into another URL).

If its counted as a URLFetch then the limit will be at least 20K queries a day.
If a game receives millions of play sessions a day, then its a good idea to set the plugin to only do the check once a week. That will spread the load on the server.

Nonetheless, even if you get to the daily limit. The game will try again the next time it run and will continue until the query is successful.

That way, if your game is set to update once a week, and the day it decides to update, google starts blocking because of the daily limit, then the game will try again the next day and the next, until it gets able to communicate with google.

Just to be sure about the limit, I'm setting up a test build that continuously queries google.
I will let it run now and will post here whats the real daily limit if there is one.

Also whats the correct way to use our own server instead of google's for this purpose.


There is no built-in implementation for updating automatically from a CSV, but you could add a script to your startup scene that downloads the file and then call:
var CSVstring = read file content
LocalizationManager.Sources[0].Import_CSV( string.Empty, CSVstring, eSpreadsheetUpdateMode.Replace );
LocalizationManager.LocalizeAll();

I see that could be useful if google have a daily limit, so I will see if in the next version I add support for external CSV files hosted on custom servers.

Thanks,
Frank

Are you :-) Give I2L 5 stars!
Are you :-( Please lets us know how to improve it!
To get the betas as soon as they are ready, check this out
Last edit: 4 years 9 months ago by Frank.
The following user(s) said Thank You: intacto

Please Log in or Create an account to join the conversation.

More
4 years 9 months ago #510 by intacto
Hi Frank,

Thanks a million for your detailed response. The csv method you offered will work perfectly for us.

Best,

Cumhur

Please Log in or Create an account to join the conversation.

More
4 years 13 hours ago - 4 years 6 hours ago #1187 by Tidhar
Hi guys,

First and foremost - I love your plugin! I'm a longtime Unity developer (way before there was even an Asset Store) and quite frankly - your plugin might be the most robust one I have used so far. Wonderful job!

As for my important question - we also might be exceeding the noted limits here. So I would like to move the spreadsheet to a specific CDN we are using.

1. Might there already be a built-in way to do so? If not, it would be of great help if you could point out (or share snippets) what steps I need to make in order to make sure I transition our project to a working solution with the CDN instead of the google usage?

2. Is there a built-in way to differ builds (or by some downloadable configuration files) between production and testing (several) environments so we can test the localization without interfering with the production?

3. How do I make sure the localization data is not fetched from the google spreadsheet I am using? Until I change the flow to use our CDN I want to only use the compiled sources and stop accessing google drive.

Thanks!
Last edit: 4 years 6 hours ago by Tidhar.

Please Log in or Create an account to join the conversation.

More
3 years 11 months ago #1188 by Frank
Hi,
Thanks!! I'm really glad you are finding the plugin useful!

we also might be exceeding the noted limits here


Are you really hitting a limit?
I ran a few tests but even with several thousand requests every hour, I wasn't hitting any limit. It may have been my own router caching the calls, that's why I'm interested in knowing if there is actually a limit as I'm not sure I2L requests fall into any of the Google quota categories.

1. Might there already be a built-in way to do so? If not, it would be of great help if you could point out (or share snippets) what steps I need to make in order to make sure I transition our project to a working solution with the CDN instead of the google usage?


There isn't at the moment a built in way to do it automatically. Nonetheless, coincidentally, I have that feature scheduled to be implemented right after some improvements I'm planning to the way Touch vs Normal localization work.
So, most likely in 2.6.4 or 2.6.5 the plugin should support hosted csv as another source for spreadsheets.

Nonetheless, there are a couple way you can implement this feature. At the basic, it is just getting the csv file using a www call and then calling Import_CSV.
You can attach the following script to one of your gameObjects, and it will download the file in the url and replace the LanguageSource with that.
using UnityEngine;
using System.Collection;

public class AutoDownloadCSV : MonoBehaviour
{
	public string url = "www.yourweb.com/I2LocData.csv";
	
	public IEnumerator Start()
	{
		var www = new WWW(url);
		yield return www;
		
		if (!string.IsNullOrEmpty(www.error))
			return;
			
		I2.Loc.LocalizationManager.Sources[0].Import_CSV(string.Empty, www.text, eSpreadsheetUpdateMode.Replace);
		I2.Loc.LocalizationManager.LocalizeAll();
	}
}

The previous code has the issue that it will download the data every time the game starts. (You may want to restrict it so that it doesn't request the data every single time)
Also, it will download the data even if it hasn't changed. So, there are two variants, if you implement your own server, you can send the loc hash and only download if its different.
The other way its to create another file (e.g. I2LManifest.txt) and place there the hash/version all of the csv, and first download that file and only request the CSV if the csv hash/version is different.

I will add those variants to my external csv implementation, but in the meantime if you are hitting a limit, you can use the script above.

2. Is there a built-in way to differ builds (or by some downloadable configuration files) between production and testing (several) environments so we can test the localization without interfering with the production?


Thats another feature I had planned, but haven't implemented yet.
For my games, I have two spreadsheets, (Development and Production). I always use the Development url, but as everytime I make a final build for the store, I manually switch the spreadsheet url.

However, thats a tedious process and prone to error. I will add that functionality to the plugin.
There will be two url for selecting the WebService and spreadsheet. Then, if the build settings have I2L_RELEASE, it will use the Production, otherwise it will fallback to Development spreadsheet.
I will also add a button to automatically move the data from the Development to the Production spreadsheet.

3. How do I make sure the localization data is not fetched from the google spreadsheet I am using? Until I change the flow to use our CDN I want to only use the compiled sources and stop accessing google drive.


If you want to stop using the google spreadsheet, you can select the I2Languages.prefab and the Google tab, select "Update Frequency" as "Never". That will disable updating from google.

If you have the game already released, and you want to stop everyone from downloading the new spreadsheet without having to update, you could delete the spreadsheet and create a new one. Players will try downloading the old spreadsheet, but the url will fail and they will skip the auto-update.


Hope that helps, and I will let you know as soon as I get the mentioned features ready for testing!
Thanks,
Frank

Are you :-) Give I2L 5 stars!
Are you :-( Please lets us know how to improve it!
To get the betas as soon as they are ready, check this out

Please Log in or Create an account to join the conversation.

More
3 years 11 months ago #1189 by Tidhar
Thanks Frank for the very detail answer.

Basically I want to make sure beforehand I will not reach the limit - we currently are not there, but I would not want to be surprised for the better with the amount of DAU and for the worst when many can reach the language data.

We sure have our own servers. I also have a download manager (for AssetBundles and files) so it could be used for flagging updates with hashes.

Cheers!

Please Log in or Create an account to join the conversation.

Time to create page: 0.128 seconds
Template by JoomlaShine