Repository navigation
Feature request: Configure options for S3 #2
Description
Activity
@scotthansonde -- thanks for following up on this.
where do these options fit in?
i don't see the term "endpoint" anywhere in the code.
a concrete question, how would this function change?
function s3NewObject (path, data, type, acl, callback, metadata) { var splitpath = s3SplitPath (path); if (type === undefined) { type = s3defaultType; } if (acl === undefined) { acl = s3defaultAcl; } var params = { ACL: acl, ContentType: type, Body: data, Bucket: splitpath.Bucket, Key: splitpath.Key, Metadata: metadata }; s3.putObject (params, function (err, data) { if (err) { console.log ("s3NewObject: error == " + err.message); s3stats.ctWriteErrors++; if (callback != undefined) { callback (err, data); } } else { s3stats.ctWrites++; s3stats.ctBytesWritten += params.Body.length; if (callback != undefined) { callback (err, data); } } }); }
@scripting As I understand it, none of the the functions in davesql.js would have to change at all. The main change would be that
const s3 = new AWS.S3 ();
becomes
const s3 = new AWS.S3 (options); //options configured by the user
Then
s3knows about the new endpoint and will use it whenever it is called.I'll try this out on an instance of publicFolder to make sure it works the way I think it does. 😃
My test with publicFolder worked.
- I installed the publicFolder helloword app per the instructions.
- After
npm install, I edited node_modules/daves3/daves3.js. I changed
const s3 = new AWS.S3 ();
to
const options = { endpoint: 's3.us-west-001.backblazeb2.com', s3ForcePathStyle: true }; const s3 = new AWS.S3 (options);
- I started the helloworld app, and it worked as expected using my bucket on Backblaze B2.
@scotthansonde -- we're going to have to make a config.json for S3 or something like that, can't have people modifying source code to make a configuration change. agree?
Reacted by Scott Hansonmaybe a better idea, add a function, s3.startup (options)
and we copy the options into a structure, and leave room for adding more options in the future.
and that way there isn't another config.json file, you have to do it in feedland's config.json and it has to be called at startup in feedland.
you can't maintain something like this and require code changes, that means if there's a bug fix or a new feature added, you have to keep track of how you modified the code! that's a guaranteed recipe for chaos. ;-)
s3 has its own configuration file,
credentials.txt, since this feature seems to be supported by amazon, maybe it will look for these values there?i won't be able to add this feature to feedland until i swing back there, so could you hold on a bit and i'll get it in the first test release of that.
s3 has its own configuration file, credentials.txt, since this feature seems to be supported by amazon, maybe it will look for these values there?
AWS has recently added configuration options for endpoints, and they work with the AWS cli, but they don't work with aws-sdk v2.
maybe a better idea, add a function, s3.startup (options).
That's exactly what I was thinking. I only modified the code in node_modules as a quick way to test. 😃
i won't be able to add this feature to feedland until i swing back there
No problem, this isn't holding me up in any way.
It would be nice to be able to configure options for S3. We would then be able to use S3 compatible services from other providers. As mentioned at scripting/feedlandInstall#7 (comment), the options for Backblaze B2 would look like:
For Cloudflare R2 it would be slightly different