Shireroth Banking Program v0.9beta now testing...

[Erik Mortis]

That was my plan..... Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

Transfer loggin inplace and working... and I just cleaned up some code.. which makes me happy... by add 6 or so lines.. I removed 20+... yay me! (well.. for the Admintool at least) and I found a solution to a problem that has been pissing me off... (and then it went away.. which worries me..) Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

Quote:alex Moved 50.12 To erik at Mon Mar 29 14:04:58 2004nick Moved 0.87 To erik at Mon Mar 29 14:05:24 2004erik Moved 45 To nick on Mon Mar 29 16:16:17 2004alex Moved 150 To nick on Mon Mar 29 16:19:49 2004erik Moved 76 To nick on Mon Mar 29 16:20:06 2004nick Moved 100 To alex on Mon Mar 29 16:20:28 2004nick Moved 100 To erik on Mon Mar 29 16:20:38 2004 Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

Oh.. and what do people think of the color scheme? hehehe This is what happens when I get module reference books.... heheheAnd Austi.. I'll have the log file editing ability up soon... view log and delete log.Ari I don't think I understand that flock() thingy.. I've never used it before... opening and closing files.. I've done that.. that's how I first did this... before DBM databases.. Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

Austi I'm emailing you the address and info for the admin tool soon.. I need to finish the new upgrades.. Erik Mortis BrookshireCommoner of Alexandretta

[Austi Scot]

Great - MiniTrade offers you 300 Erb and desires a maint. contract.

[Erik Mortis]

300... for a bank and admin tool? I was hopeing for a bit more... I believe I shall be bold and ask for...1000 €rb.. considering I'll proly lose most of it to taxes.. Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

It is finished.... Admintool and Bank.. both complete. And ready for implementation.... We got transaction logging.. including a list of reasons for transfering.. to help the Minitrade keep an eye on funds movement... the minitrade can edit account from making them adding funds.. to changing passwords..deleting accounts.. btw.. don't delete your admin account.. hehehe... or You'll need to call me in to rebuild the database..... later I'm going to add admin safety subrutines and stuff.... things like changing the admin account... ect... Erik Mortis BrookshireCommoner of Alexandretta

[Draco the Tainted]

umm... damn... cannot transfer 100 funds from erik to dev/null... or to dev\null for that matter..."Time: Mon Mar 29 22:49:50 2004Transfer FundsYour Funds are: 100Transfer To: dev/nullFunds to Transfer: 100Reason for Transfer? just because---------------------------------Time: Mon Mar 29 22:50:55 2004input contain illegal characters"--might want to check the grammar on "input contain" Heil Loki!!!
Heil H'Graa!!!

[Ari Rahikkala]

Quote:I don't think I understand that flock() thingy.. I've never used it before... opening and closing files.. I've done that.. that's how I first did this... before DBM databases..Well, you could theoretically do transfer logging with DBM, but it wouldn't be particularly useful since you need no database-like functionality for logging, but instead you *do* need your log files to be easily grepable...For the practical implementation, just use the example that's provided in the docs, its subroutines work for logfiles, too. Simply lock the file before you write anything to it and unlock after you have nothing more to write.If you're wondering why one would need locking... well, this script is running in a multiprocessing system. Consider this situation: You've got two users doing something with the bank at *almost exactly* the same time. Let's say that the process that's forked for the first user gets far enough that it's opened the log file and seeked to its end (which happens automatically if you open a file for appending). Now the *another user* comes in, preempts the first, and happens to have a complete transaction ran without being preempted. That means that the log file will have stuff appended to it, and when the system gets back to the *first* user, zir pointer that's supposed to be at the end of the file isn't there anymore. That means that the transaction log will get clobbered.flock() protects files from this by, er, locking files. This means that even if transaction 2 happens to have transaction 1 preempted at just the right time, the call to flock() will sleep until process 1 is done. Of course it's very unlikely that things like this would ever cause problems, but if the banks get busy they *might*, and you don't want to try to hunt down almost completely unreproduceable bugs... Do you really believe Project Bluebook revealed EVERYTHING the Army Air Corps knew? Do you know what REALLY happened at Roswell, New Mexico in the late 1940s? Have you even heard about the Men in Black? Do you have any friends who have `missing days'? Do you sleep with your window unlocked?Be seeing you...

[Erik Mortis]

#Transfers funds to and from acconts in order given.(funds, from, to)sub transfer { my ($funds); my ($from); my ($to); $funds = $_[0] ; $from = $_[1]; $to = $_[2]; takefunds($funds,$from); givefunds($funds,$to); logtrans($from,$funds,$to,$_[3]); } #Logs Fund transfers.sub logtrans { open(TRANLOG, '>>', 'tranlog.log'); print TRANLOG "$_[0] Moved $_[1] To $_[2] on $ltime. REASON: $_[3]\n"; close TRANLOG; }Just for reference...But wouldn't I have the same problem with the database? and wouldn't the transfer then complete with no log of it happening...? Unless I log before I move funds... Which presents it own level of problems...should a transfer fail for some reason.... than I'de have a transfer logged that didn't happen... Erik Mortis BrookshireCommoner of AlexandrettaEdited by: Erik Mortis Brookshire at: 3/30/04 10:17 am

[Erik Mortis]

So this uses the fcntl Module? I'm not sure what modules I have at my disposal on the server.. I KNOW I have CGI.pm... and I guess I can assume I have anything that comes with the perl 5.8 distrobution I guess.. but I'm not sure... Erik Mortis BrookshireCommoner of Alexandretta

[Ari Rahikkala]

I'm not sure how dbmopen/dbmclose do locking, I didn't find anything much about it in the docs, but... my "educated" guess goes like this: dbmopen and dbmclose (well, rather, the underlying database libraries) do have some form of protection from writes clobbering the physical structure of the database.However, the lack of isolation causes another problem here, and this one's a bit tougher. Consider a situation like this:1. erik has 150 erb and sends two requests to transfer 100 erb to alex's account at almost the same time2. process1 (the perl process handling the first transaction) gets to the line Quote: $efunds = ($bfunds - $mfunds); in sub takefunds and is preempted by process23. process2 completes the whole transaction. Since no money has yet been taken from erik's account in the database itself, process2 will happily assume that there's still 150 erb in erik's account (and will end with 50 erb).4. process1 gets to continue, *again* writing in the database that erik has 50 erbWorst of all, the documentation really seems to be rather scarce and nobody wants to tell me if locking and dbmopen/dbmclose go well together (probably nobody knows). Of course you can not flock() a file that you've opened with dbmopen() since you don't get an actual filehandle from it...Now, meditate on this problem for a while, for the solution isn't easy... for real, safe banking you *need* an ACID-compliant database. No, dbm (or gdbm or ndbm) won't cut it. You need MySQL with InnoDB tables or PostgreSQL, or a heavyweight commercial database.http://www.hk8.org/old_web/linux...h06_03.htm Do you really believe Project Bluebook revealed EVERYTHING the Army Air Corps knew? Do you know what REALLY happened at Roswell, New Mexico in the late 1940s? Have you even heard about the Men in Black? Do you have any friends who have `missing days'? Do you sleep with your window unlocked?Be seeing you...

[Ari Rahikkala]

Meh. I'm an idiot. Of *course* you can lock the database files. Just figure out what files your database manager uses (this should be pretty trivial), open them as you would normally open files and call flock() on them. Kills scalability because only one instance of a large part of the script will be able to run at any particular time, but it should be safe. Do you really believe Project Bluebook revealed EVERYTHING the Army Air Corps knew? Do you know what REALLY happened at Roswell, New Mexico in the late 1940s? Have you even heard about the Men in Black? Do you have any friends who have `missing days'? Do you sleep with your window unlocked?Be seeing you...

[Erik Mortis]

C:\ Perl\eg\bank>perl -cw bank.cgiNot enough arguments for lock at bank.cgi line 371, near "lock()bank.cgi had compilation errors. #Logs Fund transfers.sub logtrans { open(TRANLOG, '>>', 'tranlog.log'); lock(); print TRANLOG "$_[0] Moved $_[1] To $_[2] on $ltime. REASON: $_[3]\n"; unlock(); close TRANLOG; }#Lock file for updatesub lock { flock(TRANLOG,LOCK_EX); # and, in case someone appended # while we were waiting... seek(TRANLOG, 0, 2); }#Unlock File. sub unlock { flock(TRANLOG,LOCK_UN); }I have "use Fcntl ':flock'; # import LOCK_* constants" at the begining of the program.. but I don't know how to work with flock to begin with... But if I get it working.. I can use flock() on the DBM database? Erik Mortis BrookshireCommoner of AlexandrettaEdited by: Erik Mortis Brookshire at: 3/30/04 3:04 pm

[Erik Mortis]

well.. aside from the flock things.. I'm done with the bank...It's ready for sale. Austi you should get an email soon. Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

From the ReadMe file I'm making...Quote:III. The Future.Planned Future Features are:1. Password encyption.2. Error loging.3. Improved Security.4. Improved Error handling.5. Bug Fixes (duh!)6. Loans.7. Interest on loans and accounts.8. Prevent logging conflicts.Planned Future Features for AdminTool are:1. Prevent admin from deleting self form database.2. Change admin account..3. Set interest rates on loans.4. Set interest on accounts.I have NOOOO idea how to implement loans and interest on accounts and things.. but... that's beside the point.. I can always dream.... Mostly I'm just bored now that I finished it.. and looking for new features to create and improved security... Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

I just realized if I implement 1 way password encryption.. the admin tool can't be used to check passwords anymore.. Erik Mortis BrookshireCommoner of Alexandretta

[Ari Rahikkala]

Quote:I just realized if I implement 1 way password encryption.. the admin tool can't be used to check passwords anymore..And that's the way it should be, you know. Passwords are meant to be private.Quote:Not enough arguments for lock at bank.cgi line 371, near "lock()Hm, it seems that it's a built-in function, but only for threading perl installations... well, just change the name of the function and you ought be OK. Well, actually you probably should change that subroutine so that you could pass a reference to the filehandle that you want locked to it, but... damned if I knew how to do that.http://www.perldoc.com/perl5.6/p.../lock.htmlBTW, I'm still an idiot. It took me too much time to make the conceptual baby step that it's not the database files that need to be protected but the data in the database, which means that you can just use a lockfile. But then you'd need to use that lockfile completely atomically, which again complicates things.Therefore... I recommend this solution: Transform the program to use tie and whatever database module you want (DB_File, GDBM_File, etc. are at your disposali). Then read the documentation at search.cpan.org/~dharris/...05/Lock.pm , ignore it and do things your own way instead. Open the database file the first time you need it, put an exclusive (write) lock on it (you can do that using the tie interface) and hold it until you won't interface with the database anymore. Do you really believe Project Bluebook revealed EVERYTHING the Army Air Corps knew? Do you know what REALLY happened at Roswell, New Mexico in the late 1940s? Have you even heard about the Men in Black? Do you have any friends who have `missing days'? Do you sleep with your window unlocked?Be seeing you...Edited by: Ari Rahikkala at: 3/31/04 5:21 am

[Austi Scot]

MiniTrade is willing to pay 1000Erb - what price is the maint. contract?

[Austi Scot]

A future upgrade might be to allow an account holder to change their password. - and it would be nice if they had to put in their email address, at least to open an account. Ok. the link to the bank isn't working? - what's the deal? Edited by: Austi Scot at: 3/31/04 2:30 am

[Erik Mortis]

woot.. 1000erb.. (I can't seem to get the erb sign in Linux... odd.....oh well..)As for maintainance... I'll do bug fixes and get ready for the next version.. wich'll proly be cheaper.. by alot...(proly 100erb since it would just be an upgrade...) I'de rather not do a maintainance contract type thing....since.. that requires alot of commitment.. And I'de like to move on to another project... Erik Mortis BrookshireCommoner of Alexandretta

[Shasta Rana Wanderlust]

Noooooooooooooooo! I will no longer have the most successful company in Shireroth!...of course, you'll only be three hundred Erb ahead of me, so... Oh look, a Thing Of Beauty. Let's blow it up.--Advocate of B0O0/\/\ism : Priest of Horjin--

[Erik Mortis]

learn CGI...hehe Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

See.. the problem is ari.. I have no clue how to use this locking thing.. nor tieing.. I haven't used it befoer.. and for some reason online documentation confuses me compared to hardcopy stuff... (which I have plenty of at my access) ok.. I have the PRogramming Perl book, Learning perl and one of the module reference books.. (2nd volume...) Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

After doing some reading... flock with the LOCK_EX flag should be sufficient... as long as I use it consistantly in the bank... Erik Mortis BrookshireCommoner of Alexandretta

[Erik Mortis]

#Logs Fund transfers.sub logtrans { open(TRANLOG, '>>', 'tranlog.log'); sublock(TRANLOG); print TRANLOG "$_[0] Moved $_[1] To $_[2] on $ltime. REASON: $_[3]\n"; subunlock(TRANLOG); close TRANLOG; }#Lock file for updatesub sublock { flock($_[0] ,LOCK_EX); # and, in case someone appended # while we were waiting... seek($_[0] , 0, 2); }#Unlock File. sub subunlock { flock($_[0] ,LOCK_UN); }so.... if I just use sublock(FILEHANDLE) before I do any writing to a database...EVERYTIME I access the database.... would it be safe? As long as all programs that access the database have locking enabled in the same way? Erik Mortis BrookshireCommoner of AlexandrettaEdited by: Erik Mortis Brookshire at: 4/2/04 1:13 pm

[Erik Mortis]

#Give funds of amount to user.sub givefunds { my ($bfunds); my ($efunds); my ($user); my ($mfunds); my (%userdata); my (%userfunddata); $mfunds = $_[0] ; $bfunds = getuserfunds($_[1]); $user = $_[1]; $efunds = ($bfunds + $mfunds); dbmopen(%userfunddata, "userfunddata", 0644); $userfunddata{$user} = $efunds; dbmclose(%userfunddata); }Ok.. I really need someone to explain locking or something.. or how to use it... gdbm I think had built in locking.. but.... unfortunitly... I don't have access to it on the server... Erik Mortis BrookshireCommoner of AlexandrettaEdited by: Erik Mortis Brookshire at: 4/2/04 1:12 pm

[Ari Rahikkala]

* ari unexpectedly does some research for a change *From http://www.unixpapa.com/incnote/dbm.html we can infer that we probably do want to use locking just to protect the database files from low-level clobbering... OK, I'll put it in a clearer way:What you get if you do no locking and use a database library that does not do locking by default: Mostly it'll be fine. Then one day you'll find out that the database has suddenly become unreadable because several instances of the bank script have been working on it at the same time with no regard for each other.What you get if you do the kind of locking you're planning and use a database library that does not do locking by default: Mostly it'll be fine. Then one day you'll find that the amount of money in circulation will suddenly change from the original because different instances of the bank scripts won't know about others changing values in the database. I gave an example of this a couple of posts back. The database will always stay "healthy" in the sense that each simple read and write to/from it will be surrounded by locks.What you get if you do no locking and use a database library that does do locking by default: The same as the former.What you get if you use a single lock that spans from the first read of the database to the last write: Barring other bugs, everything in the database should stay the way it should be. Also, since now only one instance of the script will be able to use the database for anything, the bank won't be able to cope with, say, > 5000 active users (on a server of unspecified beefiness, in an economy that demands an unspecified amount of activity). I don't think that's a bad tradeoff...I don't think using tie should be so hard, the online documentation seems fine to me and in fact the example given at http://www.perldoc.com/perl5.6/p...c/tie.html involves working with a database file... however... TIMTOWDY, here are some paths that you might take:- create an empty dummy file, open it and call flock(FOO, LOCK_EX) on it in the beginning of the script, unlock it in the end- do the same with some file that you consider relevant to something. It could be a database file, the script file itself, doesn't really matter. All you need it for is ensuring that only one copy of the script ever runs- convert the script to use the tie interface, call flock() on the actual filehandle that you'll be using for I/O on the database. The only difference between the methods mentioned this far is conceptual, not practical.- learn the ways of the wizardly, port the bank to use a real RDBMS interface and do row-level locking or, if you really need performance, MVCC. That's probably beyond the scope of this project, though. Do you really believe Project Bluebook revealed EVERYTHING the Army Air Corps knew? Do you know what REALLY happened at Roswell, New Mexico in the late 1940s? Have you even heard about the Men in Black? Do you have any friends who have `missing days'? Do you sleep with your window unlocked?Be seeing you...Edited by: Ari Rahikkala at: 4/2/04 1:53 pm

[Erik Mortis]

I need to speak with my dad... Surely he knows how to do this with what we have available.. Cuase I don't really have the experience needed. and am just getting confused.. and then things make sense.. then they stop...ok.. so... I need to comply with ACID... I don't even know how to do that.... WAIT! *has idea*.. no.....the thing is I'm trying to keep things simple... so the bank works.... but this really needs to be solved..... Maybe i'll just work on preventing two people from writting to the database at the same time.. and work out ACID compliance later once I understand what I'm doing better...hmmm.. what.. if.. at the VERY beginning of the Transfer subrutine I read from a file looking for a specific phrase of something... if the file says it's safe... write to the file saying it's not safe anymore.. and preceed. that way if someone else runs the script... and it checks it'll see the database is busy and wait... than once the transfer is complete set the file to SAFE status. So the waiting script can preceed. I sense problems though... Like Ari said.. what if the give funds rutine runs before take.. wait.. is that possible? Won't it first get through take before preceeding to give?wait... what if someone reads the file RIGHT as we write saying it isn't safe... So they get past... in theory... if the first process if moving fast enough shoulnd't they not colide.. but wait.. when the 2nd process gets to take the 1st will be on give.. and collide... We are also talking about precise timing... MAYBE!Since only one script access the database. (ok.. accually that's a lie.. the admintool could access so this idea proly won't work).. make it so the script itself can only run once at a time... hmmm.... I'de have the wrap the script in another one wouldn't I?I know Ari will proly tell me to keep A.C.I.D. in mind.. but I can't take care of 2 problems at once...The bank will function with acceptable safety if I prevent 2 people from colliding... than I can move on to making it more ACID compliant as I learn how to...(and I really need to stop biting my nails.. OWWWW!)...More books.. I more books.....hehe Erik Mortis BrookshireCommoner of Alexandretta