Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2927 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1452. Min Act: -0.0000

Data Index: 194328 (The Pile)

Max Activating Token Index: 224

Text #1

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1443. Min Act: -0.0000

Data Index: 1586844 (The Pile)

Max Activating Token Index: 186

Text #2

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0170. Min Act: -0.0000

Data Index: 422947 (The Pile)

Max Activating Token Index: 168

Text #3

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1965. Min Act: -0.0000

Data Index: 154996 (The Pile)

Max Activating Token Index: 645

Text #4

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1024. Min Act: -0.0000

Data Index: 1924221 (The Pile)

Max Activating Token Index: 628

Text #5

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1849. Min Act: -0.0000

Data Index: 1896574 (The Pile)

Max Activating Token Index: 440

Text #6

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0602. Min Act: -0.0000

Data Index: 1962253 (The Pile)

Max Activating Token Index: 427

Text #7

Max Range: 1.1965. Min Range: -1.1965

Max Act: 0.9722. Min Act: -0.0000

Data Index: 1050217 (The Pile)

Max Activating Token Index: 761

Text #8

Max Range: 1.1965. Min Range: -1.1965

Max Act: 0.9675. Min Act: -0.0000

Data Index: 897200 (The Pile)

Max Activating Token Index: 859

Text #9

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0851. Min Act: -0.0000

Data Index: 1244994 (The Pile)

Max Activating Token Index: 634

Text #10

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.1175. Min Act: -0.0000

Data Index: 1368235 (The Pile)

Max Activating Token Index: 206

Text #11

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0718. Min Act: -0.0000

Data Index: 1854009 (The Pile)

Max Activating Token Index: 573

Text #12

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0213. Min Act: -0.0000

Data Index: 1327586 (The Pile)

Max Activating Token Index: 562

Text #13

Max Range: 1.1965. Min Range: -1.1965

Max Act: 0.9872. Min Act: -0.0000

Data Index: 1303292 (The Pile)

Max Activating Token Index: 675

Text #14

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0031. Min Act: -0.0000

Data Index: 963809 (The Pile)

Max Activating Token Index: 40

Text #15

Max Range: 1.1965. Min Range: -1.1965

Max Act: 0.9313. Min Act: -0.0000

Data Index: 1338542 (The Pile)

Max Activating Token Index: 768

Text #16

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0432. Min Act: -0.0000

Data Index: 448520 (The Pile)

Max Activating Token Index: 418

Text #17

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0400. Min Act: -0.0000

Data Index: 1488546 (The Pile)

Max Activating Token Index: 656

Text #18

Max Range: 1.1965. Min Range: -1.1965

Max Act: 1.0072. Min Act: -0.0000

Data Index: 294949 (The Pile)

Max Activating Token Index: 668

Text #19

Max Range: 1.1965. Min Range: -1.1965

Max Act: 0.9438. Min Act: -0.0000

Data Index: 814947 (The Pile)

Max Activating Token Index: 583