Back in the ‘burgh at Carnegie Mellon’s radio station WRCT, a really awesome poster with a baby crawling to its death in a bucket reminded you of how dangerous buckets can really be. Especially when DJing techno. I can’t work with Amazon S3 without wishing all my baby resources and folders and files would crawl in to an S3 death bucket and vanish. OMG AMAZON!
So anyhoo, to create folders instead of just files (or what appears to be folders in the awesome S3 firefox organizer), do a PUT with:
•header content_type to binary/octet-stream
•append _$folder$ to the end of the path name (as the extension)
•set the data to blank
This bitty rails code demonstrates this using the S3 plugin:
S3_FOLDER_EXT = '_$folder$'
def exists_in_S3?(file_name, bucket_name, options ={})
connect_to_S3
file_search_name = file_name.dup
file_search_name << S3_FOLDER_EXT if options[:folder]
AWS::S3::Bucket.objects(bucket_name, :prefix => file_search_name).any?
end
def create_S3_folder(file_name, bucket_name, options={})
connect_to_S3
folder = AWS::S3::S3Object.store(file_name + S3_FOLDER_EXT, '', bucket_name,
{:access => :public_read,
:content_type => "binary/octet-stream",
:cache_control => 'max-age=3600,post-check=900,pre-check=3600' }.update(options))
end
def connect_to_S3
unless AWS::S3::Base.connected?
AWS::S3::Base.establish_connection!(:access_key_id => 'blah', :secret_access_key => 'giraffe!')
end
endSo your request path would look like:
/baby_bucket?prefix=folder1/folder2_$folder$
And the output:
<ListBucketResult xmlns=”http://s3.amazonaws.com/doc/2006-03-01/”>
<Name>baby_bucket</Name>
<Prefix>folder1/folder2_$folder$</Prefix>
<Marker></Marker>
<MaxKeys>1000</MaxKeys>
<IsTruncated>false</IsTruncated>
</ListBucketResult>
At this point, you should ask yourself why you want to create folders in the first place. Because I want to stick my head in a bucket right now.
=========================================================
how can i create a folder under a bucket using boto library for amazon s3, i followed the manual, and create keys with contents with permission, metadata etc, but no where in the boto's documentation say how to create folders under bucket, or create folder under folders in bucket.
amazon-s3 boto
link|flag edited Dec 21 '09 at 12:11
skaffman
62.7k642100 asked Dec 21 '09 at 12:04
vito huang
1348
86% accept rate
it seems amazon s3 doesn't have concept of folder, some suggest creating key with name like "folder/test.txt' to get around it. i tried using firefox s3 plugin to create folder, and list all keys in boto, it outputs the folder i just created as "<Key: vitoshares,everyone_$folder$>], so how can i view/add/modify content to/from this folder? – vito huang Dec 21 '09 at 12:29
2 Answersactive
newest
votes
up vote
3
down vote
accepted Yes, thers no folder concept in S3, you can create file names like "abc/xys/uvw/123.jpg" which many S3 access tools like S3Fox show like a directory structure. But actually the its a single file in a bucket.
Amey Kanade
link|flag answered Jan 26 at 18:03
user240469
514
thanks for the answer, so i guess if i want to see the contents of a particular folder, i will need to traverse lots other unnecessary files? – vito huang Jan 27 at 23:50
The S3 API lets you query a bucket for keys with a given prefix. So you should be able to construct a request that only returns objects in your logical folder. See: docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/… – James Cooper Jul 22 at 19:58
up vote
1
down vote Hey,
Amazon S3 indeed doesn't have a concept of the folders. As you correctly mentioned you need to create keys with slash symbol to mimic that. You can use CloudBerry Explorer freeware to create folders separated by '/' rather than with $folder$ suffix.
==================================================
Create a directory on S3 - how?
Posted by: Luis Batista - Dom Digital
Posted on: Mar 9, 2010 9:23 AM Reply
This question is not answered. Helpful answers available: 2. Correct answers available: 1.
Hello,
A newbie question, how create a directory (folder) on a bucket ?
I use this code:
--------------------------------
foreach (FileInfo FI in FileList)
{
AmazonS3 clientS3 = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyId, secretAccessKey);
try
{
PutObjectRequest request = new PutObjectRequest();
request.WithFilePath(FI.DirectoryName + "
" + FI.Name)
.WithBucketName(BucketName)
.WithKey(MyDirectory + "/" + FI.Name)
.WithCannedACL(S3CannedACL.PublicRead);
S3Response response = clientS3.PutObject(request);
response.Dispose();
}
catch (AmazonS3Exception ex)
{
if (ex.ErrorCode \!= null && (ex.ErrorCode.Equals("InvalidAccessKeyId") || ex.ErrorCode.Equals("InvalidSecurity")))
{
Console.WriteLine("Please check the provided AWS Credentials.");
Console.WriteLine("If you haven't signed up for Amazon S3, please visit http://aws.amazon.com/s3");
}
else
{
Console.WriteLine("An error occurred with the message '{0}' when writing an object", ex.Message);
}
}
}
---------------------------
In "MyDirectory" variable I put the name of folder I want on bucket.
I use the firefox pluglin "S3 Organizer" to manage files on Amazon S3, if I delete the file, the directory is removed, is like the name of file include the directory name!
Any help with this situation?
Regards,
DD
Replies: 3 | Pages: 1 - Last Post: Mar 12, 2010 7:48 AM by: Colin Rhodes
Replies
Re: Create a directory on S3 - how?
Posted by: Colin Rhodes
Posted on: Mar 9, 2010 11:12 AM
in response to: Luis Batista - Dom Digital Reply
There is no such thing as directories on S3. Many S3 clients simulate folders by detecting '/' characters in the keys of the objects on S3.
Some clients will allow you to create empty objects with keys that end in a '/'. Some clients use these objects as a folder placeholder.
Re: Create a directory on S3 - how?
Posted by: Luis Batista - Dom Digital
Posted on: Mar 11, 2010 4:56 AM
in response to: Colin Rhodes Reply
Hello,
You can put a example code how create a empty object with "/" and then put objects in this "placeholder" ?
Regards,
DD
Message was edited by: Luis Batista - Dom Digital
Re: Create a directory on S3 - how?
Posted by: Colin Rhodes
Posted on: Mar 12, 2010 7:48 AM
in response to: Luis Batista - Dom Digital Reply
When I use S3, I don't create placeholder "folder" objects because I have no use for them. My main data bucket on S3 has the data broken down into sets of files associated with an ID, where the ID is the "folder".
When I want to write a data object, I simply write to "{id}/{filename}". I have no need to create a placeholder "folder" object.
The important thing to understand is that the placeholder "folder" doesn't do anything except when visualizing the structure. You don't add objects to a folder -- you just create them with the desired prefix.
That said, if you really want to create a placeholder, you could write an empty string object to the key 'myfolder/'. If an empty string doesn't work, add a single character.
==========================================================
Using .NET AWS SDK to create folder in S3 bucket
Posted by: gopinatht
Posted on: Aug 30, 2010 12:00 PM Reply
Hi All,
I apologize if this question has been asked before but I did a search and I could not find a good response.
I am trying to create a folder in a bucket in S3 using rhe AWSSDK .NET API. Here is my code:
PutObjectRequest putReq = new PutObjectRequest();
putReq.BucketName = "MyUniqueBucket";
putReq.WithKey("FromApi/");
putReq.WithContentType("binary/octet-stream");
s3Client.PutObject(putReq);
When I do this, I get the following exception:
An unhandled exception of type 'System.ArgumentException' occurred in AWSSDK.dll
Additional information: Please specify either a Filename, provide a FileStream or provide a ContentBody to PUT an object into S3.
What am I doing wrong? How can I create the folder?
Thanks in advance.
Best Regards
Gopinath
Replies: 1 | Pages: 1 - Last Post: Aug 31, 2010 12:05 AM by: Norm@AWS
Replies
Re: Using .NET AWS SDK to create folder in S3 bucket
Posted by: Norm@AWS
Posted on: Aug 31, 2010 12:05 AM
in response to: gopinatht Reply
S3 doesn't support folders in a bucket. Many tools simulate folders by adding a 0 byte object when the folder is first being "created" and then removing the 0 byte file once the first object is added to the simulated folder.
In your example you create the folder by putting a 0 byte object with an object key "FromAPI_$folder$". Then later objects would be added with the object key set to something like "FromAPI/FirstObject".
Norm
分享到:
相关推荐
这是用于 Amazon S3 Rest API 的 .NET Micro Framework 库。 您可以使用此库执行以下操作: 创建存储桶(容器、文件夹) 删除存储桶 创建对象(文件) 删除对象 由于 AWS 服务的高度安全性,库内有许多 SHA256 ...
rdiot-p022 Pi4J + Amazon S3 REST API + S3 Lambda触发器+ DynamoDB [P022]
TccInspetorAPI 这是一个Java API,提供了一种将对象上传到Amazon S3存储桶的方法。 它对应于以下体系结构的TccInspetorAPI组件: 它是在Spring Boot,Jersey和Maven开发的。 它部署在Heroku中。设置部署环境Heroku...
s3-spray.zip,s3 spraysupport通过spray客户端调用s3 rest api
以下示例演示了knox和S3 REST API的某些功能。 首先,创建一个S3客户端: var client = knox . createClient ( { key : '<api>' , secret : '<secret>' , bucket : 'learnboost' } ) ; 下面记录了更多选项,...
该库提供了两个实现Amazon S3 REST API函数的node.js模块: nox一个简单的Amazon S3客户端,支持现有存储桶的put() , get()和del()方法。 mox一个简单的S3模型,支持与nox相同的方法,可在本地驱动器上模拟Amazon ...
此类是适用于PHP 5.2.x(使用CURL)的独立Amazon S3 REST实现,支持大型文件上传,并且不需要PEAR。 这意味着它可以放入各种托管平台中并在其上运行。安装执行以下操作之一在CakePHP 2.0应用程序中使用此插件 git ...
$ s3 = new S3 ( $ awsAccessKey , $ awsSecretKey ); 静态地(例如,S3 :: getObject(...)): S3 :: setAuth ( $ awsAccessKey , $ awsSecretKey ); 对象操作 上载物件 放置文件中的对象: S3 :: putObject ...
REST 简介 REST 是英文 Representational ...目前,越来越多的 Web 服务开始采用 REST 风格设计和实现,真实世界中比较著名的 REST 服务包括:Google AJAX 搜索 API、Amazon Simple Storage Service (Amazon S3) 等。
提供了统一的接口 REST/SOAP 来统一访问任何数据 对 S3 来说,存在里面的数据就是对象名(键),和数据(值) 不限量,单个文件最高可达 5TB 高速。每个 bucket 下每秒可达 3500 PUT/COPY/POST/DELETE 或 5500 GET/...
我的技能Python HTML / CSS / JS 烧瓶本机脚本SQL SQL炼金术数据结构和算法图表树木链表堆Queue列二进制搜索最短路径算法(一点点)动态编程贪婪算法AWS EC2 弹性豆茎RDS S3 REST API Linux 的Ubuntu Bulma(CSS框架...
网络自动化中心 概述 Web 自动化中心是使用 ruby on rails 框架编写并在 ... 了解如何使用不同的 REST API(Amazon S3、Atmos REST、ViPR Management API 等) 运行一些演示来演示 API 功能 自动创建资源,例如在
詹金斯·雷斯特Java客户端建立在jclouds的顶部,可与Jenkins REST API一起使用。建立客户可以这样构建: JenkinsClient client = JenkinsClient.builder().endPoint("http://127.0.0.1:8080") // Optional. Defaults...
从驻留在S3中的文件中制作一个只读的无服务器REST端点。 对于分散系统偶尔需要的非常缓慢变化的数据很有用。 这是真实Lambda的演示版,我在工作中为任何品牌返回有效的10位数VIN,我们使用它在摄取过程中针对目标...
露天亚马逊s3-publishing-plugin 该插件用于通过 REST API 和 alfresco 的 Channel Publishing 从 Alfresco 上传和删除对象到 Amazon S3。 详情请访问:
go-ontap-rest:用于NetApp ONTAP REST API的Go库 go-ontap-rest是用于与NetApp ONTAP REST API接口的Go库。 文献资料 该库实现了ONTAP REST API的子集 请参阅 ONTAP 9.6 REST API参考》 安装 为了安装go-ontap-rest...
simples3:使用带有V4签名的REST轻松做到简单的AWS S3库 概述 SimpleS3是一个golang库,用于使用REST API调用或使用AWS Signature版本4签名的Presigned URL在S3存储桶上上载和删除对象。 安装 go get github....
Amazon S3查找和忘记 警告:在将解决方案与生产数据一起使用之前,请查阅 Amazon S3 Find and Forget是...该解决方案提供了一个Web用户界面和一个REST API,使您可以将其集成到自己的应用程序中。 请参阅以了解如何
Rest-MultiPart-API 在这里,我们创建了REST Multipart API,用于使用Spring Boot上传和下载单个/多个文件。 rest-mutipart是一个Maven项目,我们在其中使用spring boot创建了rest API,用于通过异常处理上传和下载...