libzypp  17.8.0
zypp::Fetcher Class Reference

This class allows to retrieve a group of files in a confortable way, providing some smartness that does not belong to the media layer like: More...

#include <Fetcher.h>

Inheritance diagram for zypp::Fetcher:

Classes

class  Impl
 Fetcher implementation. More...
 

Public Types

enum  Option { AutoAddContentFileIndexes = 0x0001, AutoAddChecksumsIndexes = 0x0002, AutoAddIndexes = AutoAddContentFileIndexes | AutoAddChecksumsIndexes }
 Various option flags to change behavior. More...
 

Public Member Functions

 ZYPP_DECLARE_FLAGS (Options, Option)
 
 Fetcher ()
 Default ctor. More...
 
virtual ~Fetcher ()
 Dtor. More...
 
void setOptions (Options options)
 Set the Fetcher options. More...
 
Options options () const
 Get current options. More...
 
void addIndex (const OnMediaLocation &resource)
 Adds an index containing metadata (for example checksums ) that will be retrieved and read before the job processing starts. More...
 
void enqueue (const OnMediaLocation &resource, const FileChecker &checker=FileChecker())
 Enqueue a object for transferal, they will not be transferred until start() is called. More...
 
void enqueueDigested (const OnMediaLocation &resource, const FileChecker &checker=FileChecker(), const Pathname &deltafile=Pathname())
 Enqueue a object for transferal, they will not be transferred until start() is called. More...
 
void enqueueDir (const OnMediaLocation &resource, bool recursive=false, const FileChecker &checker=FileChecker())
 Enqueue a directory. More...
 
void enqueueDigestedDir (const OnMediaLocation &resource, bool recursive=false, const FileChecker &checker=FileChecker())
 Enqueue a directory and always check for checksums. More...
 
void addCachePath (const Pathname &cache_dir)
 adds a directory to the list of directories where to look for cached files More...
 
void reset ()
 Reset the transfer (jobs) list. More...
 
void start (const Pathname &dest_dir, MediaSetAccess &media, const ProgressData::ReceiverFnc &progress=ProgressData::ReceiverFnc())
 start the transfer to a destination directory dest_dir You have to provde a media set access media to get the files from The file tree will be replicated inside this directory More...
 

Private Attributes

RWCOW_pointer< Impl_pimpl
 Pointer to implementation. More...
 

Friends

std::ostream & operator<< (std::ostream &str, const Fetcher &obj)
 

Related Functions

(Note that these are not member functions.)

std::ostream & operator<< (std::ostream &str, const Fetcher &obj)
 

Detailed Description

This class allows to retrieve a group of files in a confortable way, providing some smartness that does not belong to the media layer like:

  • Configurable local caches to retrieve already donwloaded files.
  • File checkers that can check for right checksums digital signatures, etc.
MediaSetAccess access(url, path);
Fetcher fetcher;
fetcher.enqueue( OnMediaLocation().setLocation("/somefile") );
fetcher.addCachePath("/tmp/cache")
fetcher.start( "/download-dir, access );
fetcher.reset();

To use the checkers. just create a functor implementing bool operator()(const Pathname &file)

See also
FileChecker. Pass the necessary validation data in the constructor of the functor, and pass the object to the enqueue method.
ChecksumFileChecker checker(CheckSum("sha1", "....");
fetcher.enqueue( location, checker);

If you need to use more than one checker

See also
CompositeFileChecker

Additionally, you can automatically enqueue a job with a checksum checker by using enqueueDigested which will use the OnMediaLocation checksum automatically.

location.setChecksum(CheckSum("sha1", "...."));
fetcher.enqueueDigested(location);
Note
If the checksum of the location is empty, but enqueueDigested is used, then the user will get a warning that the file has no checksum.

Additionally, Fetcher supports checking the downloaded content by using signed indexes on the remote side.

MediaSetAccess access(url, path);
Fetcher fetcher;
fetcher.addIndex(OnMediaLocation("/content"));
fetcher.enqueue( OnMediaLocation().setLocation("/somefile") );
fetcher.start( "/download-dir, access );
fetcher.reset();

Indexes are supported in CHECKSUMS format (simple text file) with checksum and file name, or content file, whith HASH SHA1 line entries.

Note
The indexes file names are relative to the directory where the index is.
libzypp-11.x: Introduction of sha256 lead to the insight that using SHA1SUMS as filename was a bad choice. New media store the checksums in a CHECKSUMS file. If a CHECKSUMS file is not present, we fall back looking for a SHA1SUMS file. The checksum type (md5,sha1,sha256) is auto detected by looking at the cheksums length. No need to somehow encode it in the filename.

Definition at line 105 of file Fetcher.h.

Member Enumeration Documentation

◆ Option

Various option flags to change behavior.

Enumerator
AutoAddContentFileIndexes 

If a content file is found, it is downloaded and read.

AutoAddChecksumsIndexes 

If a CHECKSUMS file is found, it is downloaded and read.

AutoAddIndexes 

If a content or CHECKSUMS file is found, it is downloaded and read.

Definition at line 117 of file Fetcher.h.

Constructor & Destructor Documentation

◆ Fetcher()

zypp::Fetcher::Fetcher ( )

Default ctor.

Definition at line 814 of file Fetcher.cc.

◆ ~Fetcher()

zypp::Fetcher::~Fetcher ( )
virtual

Dtor.

Definition at line 818 of file Fetcher.cc.

Member Function Documentation

◆ ZYPP_DECLARE_FLAGS()

zypp::Fetcher::ZYPP_DECLARE_FLAGS ( Options  ,
Option   
)

◆ setOptions()

void zypp::Fetcher::setOptions ( Options  options)

Set the Fetcher options.

See also
Fetcher::Options

Definition at line 821 of file Fetcher.cc.

◆ options()

Fetcher::Options zypp::Fetcher::options ( ) const

Get current options.

See also
Fetcher::Options

Definition at line 826 of file Fetcher.cc.

◆ addIndex()

void zypp::Fetcher::addIndex ( const OnMediaLocation resource)

Adds an index containing metadata (for example checksums ) that will be retrieved and read before the job processing starts.

Nothing will be transferred or checked until start() is called.

The index is relative to the media path, and the listed files too.

Indexes in the SHA1SUM format, and YaST content file

The file has to be signed or the user will be warned that the file is unsigned. You can place the signature next to the file adding the .asc extension.

If you expect the key to not to be in the target system, then you can place it next to the index using adding the .key extension.

Definition at line 851 of file Fetcher.cc.

◆ enqueue()

void zypp::Fetcher::enqueue ( const OnMediaLocation resource,
const FileChecker checker = FileChecker() 
)

Enqueue a object for transferal, they will not be transferred until start() is called.

Definition at line 857 of file Fetcher.cc.

◆ enqueueDigested()

void zypp::Fetcher::enqueueDigested ( const OnMediaLocation resource,
const FileChecker checker = FileChecker(),
const Pathname deltafile = Pathname() 
)

Enqueue a object for transferal, they will not be transferred until start() is called.

Note
As OnMediaLocation contains the digest information, a ChecksumFileChecker is automatically added to the transfer job, so make sure you don't add another one or the user could be asked twice.
Todo:
FIXME implement checker == operator to avoid this.

the optional deltafile argument describes a file that can be used for delta download algorithms. Usable files are uncompressed files or files compressed with gzip –rsyncable. Other files like rpms do not work, as the compression breaks the delta algorithm.

Definition at line 831 of file Fetcher.cc.

◆ enqueueDir()

void zypp::Fetcher::enqueueDir ( const OnMediaLocation resource,
bool  recursive = false,
const FileChecker checker = FileChecker() 
)

Enqueue a directory.

As the files to be enqueued are not known in advance, all files whose checksum can be found in some index are enqueued with checksum checking. Otherwise they are not.

Some index may provide the checksums, either by addIndex or using AutoAddIndexes flag.

Files are checked by providing a CHECKSUMS or content file listing <checksum> filename and a respective CHECKSUMS.asc/content.asc which has the signature for the checksums.

If you expect the user to not have the key of the signature either in the trusted or untrusted keyring, you can offer it as CHECKSUMS.key (or content.key)

Parameters
recursiveTrue if the complete tree should be enqueued.
Note
As checksums are read from the index, a ChecksumFileChecker is automatically added to transfer jobs having a checksum available, so make sure you don't add another one or the user could be asked twice.
The format of the file CHECKSUMS is the output of: ls | grep -v CHECKSUMS | xargs sha256sum > CHECKSUMS in each subdirectory.
Every file CHECKSUMS.* except of CHECKSUMS.(asc|key|(void)) will not be transferred and will be ignored.

Definition at line 836 of file Fetcher.cc.

◆ enqueueDigestedDir()

void zypp::Fetcher::enqueueDigestedDir ( const OnMediaLocation resource,
bool  recursive = false,
const FileChecker checker = FileChecker() 
)

Enqueue a directory and always check for checksums.

As the files to be enqueued are not known in advance, all files are enqueued with checksum checking. If the checksum of some file is not in some index, then there will be a verification warning ( DigestReport ).

Therefore some index will need to provide the checksums, either by addIndex or using AutoAddIndexes flag.

Files are checked by providing a CHECKSUMS or content file listing <checksum> filename and a respective CHECKSUMS.asc/content.asc which has the signature for the checksums.

If you expect the user to not have the key of the signature either in the trusted or untrusted keyring, you can offer it as CHECKSUMS.key (or content.key)

Parameters
recursiveTrue if the complete tree should be enqueued.
Note
As checksums are read from the index, a ChecksumFileChecker is automatically added to every transfer job, so make sure you don't add another one or the user could be asked twice.
The format of the file CHECKSUMS is the output of: ls | grep -v CHECKSUMS | xargs sha256sum > CHECKSUMS in each subdirectory.
Every file CHECKSUMS.* except of CHECKSUMS.(asc|key|(void)) will not be transferred and will be ignored.

Definition at line 843 of file Fetcher.cc.

◆ addCachePath()

void zypp::Fetcher::addCachePath ( const Pathname cache_dir)

adds a directory to the list of directories where to look for cached files

Definition at line 862 of file Fetcher.cc.

◆ reset()

void zypp::Fetcher::reset ( )

Reset the transfer (jobs) list.

Note
It does not reset the cache directory list

Definition at line 867 of file Fetcher.cc.

◆ start()

void zypp::Fetcher::start ( const Pathname dest_dir,
MediaSetAccess media,
const ProgressData::ReceiverFnc progress = ProgressData::ReceiverFnc() 
)

start the transfer to a destination directory dest_dir You have to provde a media set access media to get the files from The file tree will be replicated inside this directory

Definition at line 872 of file Fetcher.cc.

Friends And Related Function Documentation

◆ operator<< [1/2]

std::ostream& operator<< ( std::ostream &  str,
const Fetcher obj 
)
friend

Definition at line 879 of file Fetcher.cc.

◆ operator<<() [2/2]

std::ostream & operator<< ( std::ostream &  str,
const Fetcher obj 
)
related

Stream output

Definition at line 879 of file Fetcher.cc.

Member Data Documentation

◆ _pimpl

RWCOW_pointer<Impl> zypp::Fetcher::_pimpl
private

Pointer to implementation.

Definition at line 325 of file Fetcher.h.


The documentation for this class was generated from the following files: