4d27530c69
This PR updates the OffloadBinary format from version 1 to version 2,
enabling support for multiple offloading entries in a single binary.
This allows combining multiple device images into a single binary with
common global metadata while maintaining backwards compatibility with
version 1 binaries.
# Key Changes
## Binary Format Enhancements
**Version 2 Format Changes:**
- Changed from single-entry to multi-entry design
- Updated `Header` structure:
- Renamed `EntryOffset` → `EntriesOffset` (offset to entries array)
- Renamed `EntrySize` → `EntriesCount` (number of entries)
- Added `StringEntry::ValueSize` field to support explicit string value
sizes (enables non-null-terminated strings)
- Introduced `OffloadEntryFlags` enum with `OIF_Metadata` flag for
metadata-only entries (entries without binary images)
**API Changes:**
- `OffloadBinary::create()` now returns
`Expected<SmallVector<std::unique_ptr<OffloadBinary>>>` instead of
single binary
- Added optional `Index` parameter to extract specific entry:
`create(Buffer, std::optional<uint64_t> Index)`
- `OffloadBinary::write()` now accepts `ArrayRef<OffloadingImage>`
instead of single image
- Added `OffloadBinary::extractHeader()` for header extraction
**Memory Management:**
- Implemented `SharedMemoryBuffer` class to enable memory sharing across
multiple `OffloadBinary` instances from the same file
- Multiple entries from a single serialized binary share the underlying
buffer
## Testing
**Unit Tests (`unittests/Object/OffloadingTest.cpp`):**
- `checkMultiEntryBinaryExtraction`: Tests extracting all entries from a
multi-entry binary
- `checkIndexBasedExtraction`: Tests extracting specific entries by
index, including out-of-bounds validation
- `checkEdgeCases`: Tests edge cases including:
- Empty string metadata
- Empty image data
- Large string values (4KB)
**Other Tests:**
- Updated `test/ObjectYAML/Offload/multiple_members.yaml` to include
metadata-only entry
---------
Co-authored-by: Joseph Huber <huberjn@outlook.com>
63 lines
1.9 KiB
C++
63 lines
1.9 KiB
C++
//===- OffloadEmitter.cpp -------------------------------------------------===//
|
|
//
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
#include "llvm/Object/OffloadBinary.h"
|
|
#include "llvm/ObjectYAML/OffloadYAML.h"
|
|
#include "llvm/ObjectYAML/yaml2obj.h"
|
|
#include "llvm/Support/raw_ostream.h"
|
|
|
|
using namespace llvm;
|
|
using namespace OffloadYAML;
|
|
|
|
namespace llvm {
|
|
namespace yaml {
|
|
|
|
bool yaml2offload(Binary &Doc, raw_ostream &Out, ErrorHandler EH) {
|
|
SmallVector<object::OffloadBinary::OffloadingImage> Images;
|
|
for (const auto &Member : Doc.Members) {
|
|
object::OffloadBinary::OffloadingImage Image{};
|
|
if (Member.ImageKind)
|
|
Image.TheImageKind = *Member.ImageKind;
|
|
if (Member.OffloadKind)
|
|
Image.TheOffloadKind = *Member.OffloadKind;
|
|
if (Member.Flags)
|
|
Image.Flags = *Member.Flags;
|
|
|
|
if (Member.StringEntries)
|
|
for (const auto &Entry : *Member.StringEntries)
|
|
Image.StringData[Entry.Key] = Entry.Value;
|
|
|
|
SmallVector<char, 1024> Data;
|
|
raw_svector_ostream OS(Data);
|
|
if (Member.Content)
|
|
Member.Content->writeAsBinary(OS);
|
|
Image.Image = MemoryBuffer::getMemBufferCopy(OS.str());
|
|
Images.push_back(std::move(Image));
|
|
}
|
|
|
|
// Copy the data to a new buffer so we can modify the bytes directly.
|
|
auto Buffer = object::OffloadBinary::write(Images);
|
|
auto *TheHeader =
|
|
reinterpret_cast<object::OffloadBinary::Header *>(&Buffer[0]);
|
|
if (Doc.Version)
|
|
TheHeader->Version = *Doc.Version;
|
|
if (Doc.Size)
|
|
TheHeader->Size = *Doc.Size;
|
|
if (Doc.EntriesOffset)
|
|
TheHeader->EntriesOffset = *Doc.EntriesOffset;
|
|
if (Doc.EntriesCount)
|
|
TheHeader->EntriesCount = *Doc.EntriesCount;
|
|
|
|
Out.write(Buffer.begin(), Buffer.size());
|
|
|
|
return true;
|
|
}
|
|
|
|
} // namespace yaml
|
|
} // namespace llvm
|