S3FileStore

From Buni.org

S3FileStore

(Experimental)

S3FileStore operates exactly like the regular FileStore, except that it stores files to Amazon's S3 service instead of to local storage. S3 is essentially an unlimited "hard drive in the sky", allowing file sizes up to 5gb per file, and unlimted storage.

It is important to note that S3 utilizes a bucket+key mechanism to store any sort of object. Instead of a typical disk-based directory structure, each S3 account is allowed to create up to 100 buckets. Inside of each bucket, a key is stored that maps to a specific object. The bucket-name+key added together create a unique identifier that maps to a given file object.

For example, a user wishing to store the file "test.txt" would need to select an S3-unique bucket name (e.g., meldware). The actual binary data would then be stored in 'meldware/test.txt'.

S3 keys can contain any character, and can be up to 1024 characters long. Thus, keys can be used to simulate a directory structure. For example, the following is a valid key: 'temp/files/test.txt'. Storing this sample key would create a bucket+key mapping of 'meldware/temp/files/test.txt'.

Like the regular Filestore, [b]S3FileStore[/b] uses a series of directories to store data, each with a unique key. For instance item 1005 would be stored with a key of '/1/0/0/X5', in whichever particular S3 bucket you designate.

S3Filestore does not put out its own separate distributions. You can acquire it by downloading Meldware and getting it from the server/default/deploy/mail.ear/mail.sar/buni-s3filestore-xxx.jar or by building it from the source (follow these instructions replacing the module name with "s3filestore"). You can browse the source here.

Configuring S3FileStore

To enable the S3FileStore, it is necessary to edit several files.

1.) Edit Meldware's application.xml
Open the file "{JBOSS_HOME}/server/{MELDWARE_ROOT}/deploy/mail.ear/META-INF/application.xml" and add the following module underneath the <application> tag:

   <module>
   	<java>buni-s3filestore-0.1.jar</java>
   </module>

2.) Comment out existing FileStore and MailBody Managers
Open the file "{JBOSS_HOME}/server/{MELDWARE_ROOT}/deploy/mail.ear/mailjmx.sar/META-INF/jboss-service.xml" and comment out the following modules (if they exist):

a.) 'meldware.mail:type=MailServices,name=FileStore'
b.) 'meldware.mail:type=MailServices,name=PostgreSQLStore'
c.) 'meldware.mail:type=MailServices,name=MailBodyManager'

3.) Add the S3 XML Entries below.
Open the file "{JBOSS_HOME}/server/{MELDWARE_ROOT}/deploy/mail.ear/mailjmx.sar/META-INF/jboss-service.xml" and add out the following modules:

<mbean code="org.buni.meldware.mail.store.s3.S3FileStoreServiceImpl"

 name="meldware.mail:type=MailServices,name=FileStore" 
 xmbean-dd="META-INF/S3FileStore-xmbean.xml"> 
 <attribute name="DefaultS3Host">s3.amazonaws.com</attribute>
 <attribute name="UseSSL">true</attribute>
 <attribute name="AwsSystemAccessKey">ENTER YOUR KEY</attribute>
 <attribute name="AwsSystemSecretAccessKey">ENTER YOUR SECRET KEY</attribute>
 <attribute name="DefaultBucket">ENTER YOUR DEFAULT BUCKET</attribute>
 <attribute name="DefaultPath">mail</attribute>
 <attribute name="NumRetries">3</attribute>
 <attribute name="RetrySleep">3000</attribute>
 	
 <attribute name="MaxS3FileSize">5368709120</attribute>
 <attribute name="PageSize">131072</attribute>
 <attribute name="Compress">true</attribute>
 <attribute name="CompressBufferSize">65535</attribute>
 <attribute name="StartIndex">0</attribute>
 <attribute name="Hashed">false</attribute>

</mbean>
<mbean code="org.buni.meldware.mail.message.MailBodyManagerImpl"

 name="meldware.mail:type=MailServices,name=MailBodyManager">
 <attribute name="UseStore">true</attribute>
 <depends optional-attribute-name="Store" proxy-type="attribute">
   meldware.mail:type=MailServices,name=FileStore
 </depends>

</mbean>

4.) Amazon S3 Service Note
If you haven't already, you will need an Amazon web-services account, and then enable it (on Amazon.com) to use S3. See http://www.amazon.com/s3. Once registered, you'll need to enter your S3 credentials into the above jboss-server.xml file.

5.) Restart the Server.

Search

Buni.org

Personal tools

Toolbox