home *** CD-ROM | disk | FTP | other *** search
- Path: sparky!uunet!dove!dove.nist.gov!przemek
- From: przemek@rrdstrad.nist.gov (Przemek Klosowski)
- Newsgroups: comp.lang.perl
- Subject: How to limit Perl memory usage?
- Message-ID: <PRZEMEK.93Jan5181833@rrdstrad.nist.gov>
- Date: 5 Jan 93 23:18:33 GMT
- Sender: news@dove.nist.gov
- Organization: U. of Maryland/NIST
- Lines: 87
-
-
- Hello!
-
- I wrote a Perl script to archive data from across the network (from
- non-unix machines). The script uses FTP to get the remote files,
- calling G. Spafford's ftplib.pl library.
-
- The program seems to have a memory leak: it grows by 100 KB per each
- file transfered (and I may have several hundred files per run).
-
- This is a core of the script (%getlist contains a list of candidates for
- transfer):
-
- while (($key,$val) = each %getlist) {
-
- # <prepare source and destination names>
-
- if (&check_file($fullname)) {
- &ftp'get($fullname, $unixpath) || do # '
- { #
- warn &ftp'error; # '
- &booboo("Coudn't get file $host::$fullname");
- &ftp'close; # 'shut down ftp
- next COMPUTER; # give up on this computer for now
- }; #
- } else {
- warn "Couldn't find file $host::$fullname\n";
- }
- $BCKLST{$key} = $val;
- }
-
- where routine check_file checks if the remote file exists
-
- sub check_file { # does the unique file exist on the FTP host?
- local($file)=@_[0];
- local(@listnames);
- (@listnames = &ftp'list($file)) || (warn &ftp'error, return undef);
- return pop @listnames if( $#listnames == 0 ); # worry if more than one...
- return undef; # otherwise
- }
-
- The only place that I can think of that may leak memory is the ftp'do_ftp_listing
- routine that I include below: it seems to allocate a global array @ftp_list
- and pass it as a result. I have those questions:
-
- - is this consistent with my observed memory growth? I mean, typically
- there would be only one file returned by ftp'list(); why would the list
- take 100KB?
- If not, what else do I miss?
-
- If yes,
-
- - is there a way of deallocating the storage created in do_ftp_listing?
-
-
-
- sub list { ## Public
- &do_ftp_listing("nlst", @_);
- }
- sub do_ftp_listing { ## Private
- local(@lcmd) = @_;
- @ftp_list = ();
- $ftp_trans_flag = 0;
-
- return undef unless &do_open_dport;
-
- return undef unless &do_ftp_cmd(150, @lcmd);
- do { # Following is grotty, but chat2 makes us do it
- &chat'expect($Data_handle, 30,
- "(.*)\r?\n", 'push(@ftp_list, $1)',
- "EOF", '$ftp_trans_flag = 1');
- } until $ftp_trans_flag;
-
- &chat'close($Data_handle);
- return undef unless &do_ftp_cmd(226);
-
- grep(y/\r\n//d, @ftp_list);
- @ftp_list;
- }
-
- --
- przemek klosowski (przemek@rrdstrad.nist.gov)
- Reactor Division (bldg. 235), E111
- National Institute of Standards and Technology
- Gaithersburg, MD 20899, USA
-
- (301) 975 6249
-