Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4082 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1767. Min Range: -1.1767

Max Act: 1.1724. Min Act: -0.0000

Data Index: 584863 (The Pile)

Max Activating Token Index: 379

Text #1

Max Range: 1.1767. Min Range: -1.1767

Max Act: 1.1767. Min Act: -0.0000

Data Index: 1438432 (The Pile)

Max Activating Token Index: 736

Text #2

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9651. Min Act: -0.0000

Data Index: 1695471 (The Pile)

Max Activating Token Index: 412

Text #3

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9227. Min Act: -0.0000

Data Index: 1306044 (The Pile)

Max Activating Token Index: 233

Text #4

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7497. Min Act: -0.0000

Data Index: 1430991 (The Pile)

Max Activating Token Index: 314

Text #5

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8887. Min Act: -0.0000

Data Index: 65924 (The Pile)

Max Activating Token Index: 441

Text #6

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7087. Min Act: -0.0000

Data Index: 10660 (The Pile)

Max Activating Token Index: 96

Text #7

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9104. Min Act: -0.0000

Data Index: 1805307 (The Pile)

Max Activating Token Index: 77

Text #8

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.8420. Min Act: -0.0000

Data Index: 612722 (The Pile)

Max Activating Token Index: 466

Text #9

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.5966. Min Act: -0.0000

Data Index: 1983606 (The Pile)

Max Activating Token Index: 747

Text #10

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7500. Min Act: -0.0000

Data Index: 87263 (The Pile)

Max Activating Token Index: 249

Text #11

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.9513. Min Act: -0.0000

Data Index: 1899814 (The Pile)

Max Activating Token Index: 716

Text #12

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7328. Min Act: -0.0000

Data Index: 1318198 (The Pile)

Max Activating Token Index: 645

Text #13

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7114. Min Act: -0.0000

Data Index: 1934516 (The Pile)

Max Activating Token Index: 776

Text #14

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7673. Min Act: -0.0000

Data Index: 24279 (The Pile)

Max Activating Token Index: 579

Text #15

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7948. Min Act: -0.0000

Data Index: 1607725 (The Pile)

Max Activating Token Index: 794

Text #16

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7144. Min Act: -0.0000

Data Index: 1004607 (The Pile)

Max Activating Token Index: 549

Text #17

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7853. Min Act: -0.0000

Data Index: 1493261 (The Pile)

Max Activating Token Index: 420

Text #18

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.7154. Min Act: -0.0000

Data Index: 1827486 (The Pile)

Max Activating Token Index: 837

Text #19

Max Range: 1.1767. Min Range: -1.1767

Max Act: 0.6657. Min Act: -0.0000

Data Index: 584570 (The Pile)

Max Activating Token Index: 667