data/TWiki/TWikiStoreDotPm.txt
author Colas Nahaboo <colas@nahaboo.net>
Sat, 26 Jan 2008 15:50:53 +0100
changeset 0 414e01d06fd5
child 2 7bc60a767fa4
permissions -rw-r--r--
RELEASE 4.2.0 freetown
     1 ---+ Package =TWiki::Store=
     2 
     3 This module hosts the generic storage backend. This module provides
     4 the interface layer between the "real" store provider - which is hidden
     5 behind a handler - and the rest of the system. it is responsible for
     6 checking for topic existance, access permissions, and all the other
     7 general admin tasks that are common to all store implementations.
     8 
     9 This module knows nothing about how the data is actually _stored_ -
    10 that knowledge is entirely encapsulated in the handlers.
    11 
    12 The general contract for methods in the class requires that errors
    13 are signalled using exceptions. TWiki::AccessControlException is
    14 used for access control exceptions, and Error::Simple for all other
    15 types of error.
    16 
    17 
    18 %TOC%
    19 
    20 ---++ ClassMethod *new* <tt>($session)</tt>
    21 
    22 Construct a Store module, linking in the chosen sub-implementation.
    23 
    24 
    25 
    26 ---++ ObjectMethod *finish* <tt>()</tt>
    27 Break circular references.
    28 
    29 
    30 
    31 ---++ ObjectMethod *readTopic* <tt>($user,$web,$topic,$version) -> ($metaObject,$text)</tt>
    32 
    33 Reads the given version of a topic and it's meta-data. If the version
    34 is undef, then read the most recent version. The version number must be
    35 an integer, or undef for the latest version.
    36 
    37 if $user is defined, view permission will be required for the topic
    38 read to be successful.  Access control violations are flagged by a
    39 TWiki::AccessControlException. Permissions are checked for the user
    40 name passed in.
    41 
    42 If the topic contains a web specification (is of the form Web.Topic) the
    43 web specification will override whatever is passed in $web.
    44 
    45 The metadata and topic text are returned separately, with the metadata in a
    46 TWiki::Meta object.  (The topic text is, as usual, just a string.)
    47 
    48 
    49 
    50 ---++ ObjectMethod *_findAttachments* <tt>($session,$web,$topic,$knownAttachments) -> @attachmentsFoundInPub</tt>
    51 
    52 Synchronise the attachment list with what's actually on disk Returns an ARRAY
    53 of FILEATTACHMENTs. These can be put in the new meta using
    54 meta->put('FILEATTACHMENTS', $tree) 
    55 
    56 This function is only called when the AutoAttachPubFiles configuration option is set.
    57 
    58 IDEA On Windows machines where the underlying filesystem can store arbitary
    59 meta data against files, this might replace/fulfil the COMMENT purpose
    60 
    61 TODO consider logging when things are added to metadata
    62 
    63 
    64 
    65 ---++ ObjectMethod *readTopicRaw* <tt>($user,$web,$topic,$version) -> $topicText</tt>
    66 
    67 Reads the given version of a topic, without separating out any embedded
    68 meta-data. If the version is undef, then read the most recent version.
    69 The version number must be an integer or undef.
    70 
    71 If $user is defined, view permission will be required for the topic
    72 read to be successful.  Access control violations are flagged by a
    73 TWiki::AccessControlException. Permissions are checked for the user
    74 name passed in.
    75 
    76 If the topic contains a web specification (is of the form Web.Topic) the
    77 web specification will override whatever is passed in $web.
    78 
    79 SMELL: DO NOT CALL THIS METHOD UNLESS YOU HAVE NO CHOICE. This method breaks
    80 encapsulation of the store, as it assumes meta is stored embedded in the text.
    81 Other implementors of store will be forced to insert meta-data to ensure
    82 correct operation of View raw=debug and the 'repRev' mode of Edit.
    83 
    84 $web and $topic _must_ be untainted.
    85 
    86 
    87 
    88 ---++ ObjectMethod *moveAttachment* <tt>($oldWeb,$oldTopic,$oldAttachment,$newWeb,$newTopic,$newAttachment,$user)</tt>
    89 
    90 Move an attachment from one topic to another.
    91 
    92 The caller to this routine should check that all topics are valid.
    93 
    94 All parameters must be defined, and must be untainted.
    95 
    96 
    97 
    98 ---++ ObjectMethod *getAttachmentStream* <tt>($user,$web,$topic,$attName) -> \*STREAM</tt>
    99 
   100    * =$user= - the user doing the reading, or undef if no access checks
   101    * =$web= - The web
   102    * =$topic= - The topic
   103    * =$attName= - Name of the attachment
   104 
   105 Open a standard input stream from an attachment.
   106 
   107 If $user is defined, view permission will be required for the topic
   108 read to be successful.  Access control violations and errors will
   109 cause exceptions to be thrown.
   110 
   111 Permissions are checked for the user name passed in.
   112 
   113 
   114 
   115 ---++ ObjectMethod *getAttachmentList* <tt>($web,$topic)</tt>
   116 
   117 returns @($attachmentName => [stat]) for any given web, topic
   118 
   119 
   120 
   121 ---++ ObjectMethod *attachmentExists* <tt>($web,$topic,$att) -> $boolean</tt>
   122 
   123 Determine if the attachment already exists on the given topic
   124 
   125 
   126 
   127 ---++ ObjectMethod *_removeAutoAttachmentsFromMeta* <tt></tt>
   128 
   129 This is where we are going to remove from meta any entry that is marked as an automatic attachment.
   130 
   131 
   132 
   133 ---++ ObjectMethod *moveTopic* <tt>($oldWeb,$oldTopic,$newWeb,$newTopic,$user)</tt>
   134 
   135 All parameters must be defined and must be untainted.
   136 
   137 
   138 
   139 ---++ ObjectMethod *moveWeb* <tt>($oldWeb,$newWeb,$user)</tt>
   140 
   141 Move a web.
   142 
   143 All parrameters must be defined and must be untainted.
   144 
   145 
   146 
   147 ---++ ObjectMethod *readAttachment* <tt>($user,$web,$topic,$attachment,$theRev) -> $text</tt>
   148 
   149 Read the given version of an attachment, returning the content.
   150 
   151 View permission on the topic is required for the
   152 read to be successful.  Access control violations are flagged by a
   153 TWiki::AccessControlException. Permissions are checked for the user
   154 passed in.
   155 
   156 If $theRev is not given, the most recent rev is assumed.
   157 
   158 
   159 
   160 ---++ ObjectMethod *getRevisionNumber* <tt>($web,$topic,$attachment) -> $integer</tt>
   161 
   162 Get the revision number of the most recent revision. Returns
   163 the integer revision number or '' if the topic doesn't exist.
   164 
   165 WORKS FOR ATTACHMENTS AS WELL AS TOPICS
   166 
   167 
   168 
   169 ---++ ObjectMethod *getWorkArea* <tt>($key) -> $directorypath</tt>
   170 
   171 Gets a private directory uniquely identified by $key. The directory is
   172 intended as a work area for plugins. The directory will exist.
   173 
   174 
   175 
   176 ---++ ObjectMethod *getRevisionDiff* <tt>($user,$web,$topic,$rev1,$rev2,$contextLines) -> \@diffArray</tt>
   177 
   178 Return reference to an array of [ diffType, $right, $left ]
   179    * =$user= - the user id, or undef to suppress access control checks
   180    * =$web= - the web
   181    * =$topic= - the topic
   182    * =$rev1= Integer revision number
   183    * =$rev2= Integer revision number
   184    * =$contextLines= - number of lines of context required
   185 
   186 
   187 
   188 ---++ ObjectMethod *getRevisionInfo* <tt>($web,$topic,$rev,$attachment) -> ($date,$user,$rev,$comment)</tt>
   189 
   190 Get revision info of a topic.
   191    * =$web= Web name, optional, e.g. ='Main'=
   192    * =$topic= Topic name, required, e.g. ='TokyoOffice'=
   193    * =$rev= revision number. If 0, undef, or out-of-range, will get info about the most recent revision.
   194    * =$attachment= attachment filename; undef for a topic
   195 Return list with: ( last update date, last user id, =
   196 | $date | in epochSec |
   197 | $user | user *object* |
   198 | $rev | the revision number |
   199 | $comment | WHAT COMMENT? |
   200 e.g. =( 1234561, 'phoeny', 5, 'no comment' )
   201 
   202 NOTE NOTE NOTE if you are working within the TWiki code DO NOT USE THIS
   203 FUNCTION FOR GETTING REVISION INFO OF TOPICS - use
   204 TWiki::Meta::getRevisionInfo instead. This is essential to allow clean
   205 transition to a topic object model later, and avoids the risk of confusion
   206 coming from meta and Store revision information being out of step.
   207 (it's OK to use it for attachments)
   208 
   209 
   210 
   211 ---++ StaticMethod *dataEncode* <tt>($uncoded) -> $coded</tt>
   212 
   213 Encode meta-data fields, escaping out selected characters. The encoding
   214 is chosen to avoid problems with parsing the attribute values, while
   215 minimising the number of characters encoded so searches can still work
   216 (fairly) sensibly.
   217 
   218 The encoding has to be exported because TWiki (and plugins) use
   219 encoded field data in other places e.g. RDiff, mainly as a shorthand
   220 for the properly parsed meta object. Some day we may be able to
   221 eliminate that....
   222 
   223 
   224 
   225 ---++ StaticMethod *dataDecode* <tt>($encoded) -> $decoded</tt>
   226 
   227 Decode escapes in a string that was encoded using dataEncode
   228 
   229 The encoding has to be exported because TWiki (and plugins) use
   230 encoded field data in other places e.g. RDiff, mainly as a shorthand
   231 for the properly parsed meta object. Some day we may be able to
   232 eliminate that....
   233 
   234 
   235 
   236 ---++ ObjectMethod *saveTopic* <tt>($user,$web,$topic,$text,$meta,$options)</tt>
   237 
   238    * =$user= - user doing the saving (object)
   239    * =$web= - web for topic
   240    * =$topic= - topic to atach to
   241    * =$text= - topic text
   242    * =$meta= - topic meta-data
   243    * =$options= - Ref to hash of options
   244 =$options= may include:
   245 | =dontlog= | don't log this change in twiki log |
   246 | =hide= | if the attachment is to be hidden in normal topic view |
   247 | =comment= | comment for save |
   248 | =file= | Temporary file name to upload |
   249 | =minor= | True if this is a minor change (used in log) |
   250 | =savecmd= | Save command |
   251 | =forcedate= | grr |
   252 | =unlock= | |
   253 
   254 Save a new revision of the topic, calling plugins handlers as appropriate.
   255 
   256 
   257 
   258 ---++ ObjectMethod *saveAttachment* <tt>($web,$topic,$attachment,$user,$opts)</tt>
   259 
   260    * =$user= - user doing the saving
   261    * =$web= - web for topic
   262    * =$topic= - topic to atach to
   263    * =$attachment= - name of the attachment
   264    * =$opts= - Ref to hash of options
   265 =$opts= may include:
   266 | =dontlog= | don't log this change in twiki log |
   267 | =comment= | comment for save |
   268 | =hide= | if the attachment is to be hidden in normal topic view |
   269 | =stream= | Stream of file to upload |
   270 | =file= | Name of a file to use for the attachment data. ignored is stream is set. |
   271 | =filepath= | Client path to file |
   272 | =filesize= | Size of uploaded data |
   273 | =filedate= | Date |
   274 | =tmpFilename= | Pathname of the server file the stream is attached to. Required if stream is set. |
   275 
   276 Saves a new revision of the attachment, invoking plugin handlers as
   277 appropriate.
   278 
   279 If file is not set, this is a properties-only save.
   280 
   281 
   282 
   283 ---++ ObjectMethod *repRev* <tt>($user,$web,$topic,$text,$meta,$options)</tt>
   284 
   285 Replace last (top) revision with different text.
   286 
   287 Parameters and return value as saveTopic, except
   288    * =$options= - as for saveTopic, with the extra option:
   289       * =timetravel= - if we want to force the deposited revision to look as much like the revision specified in =$rev= as possible.
   290       * =operation= - set to the name of the operation performing the save. This is used only in the log, and is normally =cmd= or =save=. It defaults to =save=.
   291 
   292 Used to try to avoid the deposition of 'unecessary' revisions, for example
   293 where a user quickly goes back and fixes a spelling error.
   294 
   295 Also provided as a means for administrators to rewrite history (timetravel).
   296 
   297 It is up to the store implementation if this is different
   298 to a normal save or not.
   299 
   300 
   301 
   302 ---++ ObjectMethod *delRev* <tt>($user,$web,$topic,$text,$meta,$options)</tt>
   303 
   304 Parameters and return value as saveTopic.
   305 
   306 Provided as a means for administrators to rewrite history.
   307 
   308 Delete last entry in repository, restoring the previous
   309 revision.
   310 
   311 It is up to the store implementation whether this actually
   312 does delete a revision or not; some implementations will
   313 simply promote the previous revision up to the head.
   314 
   315 
   316 
   317 ---++ ObjectMethod *lockTopic* <tt>($web,$topic)</tt>
   318 
   319 Grab a topic lock on the given topic. A topic lock will cause other
   320 processes that also try to claim a lock to block. A lock has a
   321 maximum lifetime of 2 minutes, so operations on a locked topic
   322 must be completed within that time. You cannot rely on the
   323 lock timeout clearing the lock, though; that should always
   324 be done by calling unlockTopic. The best thing to do is to guard
   325 the locked section with a try..finally clause. See man Error for more info.
   326 
   327 Topic locks are used to make store operations atomic. They are
   328 _note_ the locks used when a topic is edited; those are Leases
   329 (see =getLease=)
   330 
   331 
   332 
   333 ---++ ObjectMethod *unlockTopic* <tt>($user,$web,$topic)</tt>
   334 
   335 Release the topic lock on the given topic. A topic lock will cause other
   336 processes that also try to claim a lock to block. It is important to
   337 release a topic lock after a guard section is complete. This should
   338 normally be done in a 'finally' block. See man Error for more info.
   339 
   340 Topic locks are used to make store operations atomic. They are
   341 _note_ the locks used when a topic is edited; those are Leases
   342 (see =getLease=)
   343 
   344 
   345 
   346 ---++ ObjectMethod *webExists* <tt>($web) -> $boolean</tt>
   347 
   348 Test if web exists
   349    * =$web= - Web name, required, e.g. ='Sandbox'=
   350 
   351 A web _has_ to have a preferences topic to be a web.
   352 
   353 
   354 
   355 ---++ ObjectMethod *topicExists* <tt>($web,$topic) -> $boolean</tt>
   356 
   357 Test if topic exists
   358    * =$web= - Web name, optional, e.g. ='Main'=
   359    * =$topic= - Topic name, required, e.g. ='TokyoOffice'=, or ="Main.TokyoOffice"=
   360 
   361 
   362 
   363 ---++ ObjectMethod *getTopicParent* <tt>($web,$topic) -> $string</tt>
   364 
   365 Get the name of the topic parent. Needs to be fast because
   366 of use by Render.pm.
   367 
   368 
   369 
   370 ---++ ObjectMethod *getTopicLatestRevTime* <tt>($web,$topic) -> $epochSecs</tt>
   371 
   372 Get an approximate rev time for the latest rev of the topic. This method
   373 is used to optimise searching. Needs to be as fast as possible.
   374 
   375 
   376 
   377 ---++ ObjectMethod *eachChange* <tt>($web,$time) -> $iterator</tt>
   378 
   379 Get an iterator over the list of all the changes in the given web between
   380 =$time= and now. $time is a time in seconds since 1st Jan 1970, and is not
   381 guaranteed to return any changes that occurred before (now - 
   382 {Store}{RememberChangesFor}). Changes are returned in most-recent-first
   383 order.
   384 
   385 
   386 
   387 ---++ ObjectMethod *getTopicNames* <tt>($web) -> @topics</tt>
   388 
   389 Get list of all topics in a web
   390    * =$web= - Web name, required, e.g. ='Sandbox'=
   391 Return a topic list, e.g. =( 'WebChanges',  'WebHome', 'WebIndex', 'WebNotify' )=
   392 
   393 
   394 
   395 ---++ ObjectMethod *getListOfWebs* <tt>($filter) -> @webNames</tt>
   396 
   397 Gets a list of webs, filtered according to the spec in the $filter,
   398 which may include one of:
   399    1 'user' (for only user webs)
   400    2 'template' (for only template webs)
   401 $filter may also contain the word 'public' which will further filter
   402 webs on whether NOSEARCHALL is specified for them or not.
   403 'allowed' filters out webs that the user is denied access to by a *WEBVIEW.
   404 
   405 If $TWiki::cfg{EnableHierarchicalWebs} is set, will also list
   406 sub-webs recursively.
   407 
   408 
   409 
   410 ---++ ObjectMethod *createWeb* <tt>($user,$newWeb,$baseWeb,$opts)</tt>
   411 
   412 $newWeb is the name of the new web.
   413 
   414 $baseWeb is the name of an existing web (a template web). If the
   415 base web is a system web, all topics in it
   416 will be copied into the new web. If it is a normal web, only topics starting
   417 with 'Web' will be copied. If no base web is specified, an empty web
   418 (with no topics) will be created. If it is specified but does not exist,
   419 an error will be thrown.
   420 
   421 $opts is a ref to a hash that contains settings to be modified in
   422 the web preferences topic in the new web.
   423 
   424 
   425 
   426 ---++ ObjectMethod *removeWeb* <tt>($user,$web)</tt>
   427 
   428    * =$user= - user doing the removing (for the history)
   429    * =$web= - web being removed
   430 
   431 Destroy a web, utterly. Removed the data and attachments in the web.
   432 
   433 Use with great care!
   434 
   435 The web must be a known web to be removed this way.
   436 
   437 
   438 
   439 ---++ ObjectMethod *getDebugText* <tt>($meta,$text) -> $text</tt>
   440 
   441 Generate a debug text form of the text/meta, for use in debug displays,
   442 by annotating the text with meta informtion.
   443 
   444 
   445 
   446 ---++ ObjectMethod *cleanUpRevID* <tt>($rev) -> $integer</tt>
   447 
   448 Cleans up (maps) a user-supplied revision ID and converts it to an integer
   449 number that can be incremented to create a new revision number.
   450 
   451 This method should be used to sanitise user-provided revision IDs.
   452 
   453 
   454 
   455 ---++ ObjectMethod *copyTopic* <tt>($user,$fromweb,$fromtopic,$toweb,$totopic)</tt>
   456 
   457 Copy a topic and all it's attendant data from one web to another.
   458 
   459 SMELL: Does not fix up meta-data!
   460 
   461 
   462 
   463 ---++ ObjectMethod *searchMetaData* <tt>($params) -> $text</tt>
   464 
   465 Search meta-data associated with topics. Parameters are passed in the $params hash,
   466 which may contain:
   467 | =type= | =topicmoved=, =parent= or =field= |
   468 | =topic= | topic to search for, for =topicmoved= and =parent= |
   469 | =name= | form field to search, for =field= type searches. May be a regex. |
   470 | =value= | form field value. May be a regex. |
   471 | =title= | Title prepended to the returned search results |
   472 | =default= | defualt value if there are no results |
   473 | =web= | web to search in, default is all webs |
   474 | =format= | string for custom formatting results |
   475 The idea is that people can search for meta-data values without having to be
   476 aware of how or where meta-data is stored.
   477 
   478 SMELL: should be replaced with a proper SQL-like search, c.f. Plugins.DBCacheContrib.
   479 
   480 
   481 
   482 ---++ ObjectMethod *searchInWebMetaData* <tt>($query,$web,\@topics) -> \%matches</tt>
   483 
   484 Search for a meta-data expression in the content of a web. =$query= must be a =TWiki::Query= object.
   485 
   486 Returns a reference to a hash that maps the names of topics that all matched
   487 to the result of the query expression (e.g. if the query expression is
   488 'TOPICPARENT.name' then you will get back a hash that maps topic names
   489 to their parent.
   490 
   491 
   492 
   493 ---++ ObjectMethod *searchInWebContent* <tt>($searchString,$web,\@topics,\%options) -> \%map</tt>
   494 
   495 Search for a string in the content of a web. The search must be over all
   496 content and all formatted meta-data, though the latter search type is
   497 deprecated (use searchMetaData instead).
   498 
   499    * =$searchString= - the search string, in egrep format if regex
   500    * =$web= - The web to search in
   501    * =\@topics= - reference to a list of topics to search
   502    * =\%options= - reference to an options hash
   503 The =\%options= hash may contain the following options:
   504    * =type= - if =regex= will perform a egrep-syntax RE search (default '')
   505    * =casesensitive= - false to ignore case (defaulkt true)
   506    * =files_without_match= - true to return files only (default false)
   507 
   508 The return value is a reference to a hash which maps each matching topic
   509 name to a list of the lines in that topic that matched the search,
   510 as would be returned by 'grep'. If =files_without_match= is specified, it will
   511 return on the first match in each topic (i.e. it will return only one
   512 match per topic, and will not return matching lines).
   513 
   514 
   515 
   516 ---++ ObjectMethod *getRevisionAtTime* <tt>($web,$topic,$time) -> $rev</tt>
   517 
   518    * =$web= - web for topic
   519    * =$topic= - topic
   520    * =$time= - time (in epoch secs) for the rev
   521 
   522 Get the revision number of a topic at a specific time.
   523 Returns a single-digit rev number or undef if it couldn't be determined
   524 (either because the topic isn't that old, or there was a problem)
   525 
   526 
   527 
   528 ---++ ObjectMethod *getLease* <tt>($web,$topic) -> $lease</tt>
   529 
   530    * =$web= - web for topic
   531    * =$topic= - topic
   532 
   533 If there is an lease on the topic, return the lease, otherwise undef.
   534 A lease is a block of meta-information about a topic that can be
   535 recovered (this is a hash containing =user=, =taken= and =expires=).
   536 Leases are taken out when a topic is edited. Only one lease
   537 can be active on a topic at a time. Leases are used to warn if
   538 another user is already editing a topic.
   539 
   540 
   541 
   542 ---++ ObjectMethod *setLease* <tt>($web,$topic,$user,$length)</tt>
   543 
   544 Take out an lease on the given topic for this user for $length seconds.
   545 
   546 See =getLease= for more details about Leases.
   547 
   548 
   549 
   550 ---++ ObjectMethod *clearLease* <tt>($web,$topic)</tt>
   551 
   552 Cancel the current lease.
   553 
   554 See =getLease= for more details about Leases.
   555 
   556 
   557 
   558 ---++ ObjectMethod *removeSpuriousLeases* <tt>($web)</tt>
   559 
   560 Remove leases that are not related to a topic. These can get left behind in
   561 some store implementations when a topic is created, but never saved.
   562 
   563