Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2426 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.2046. Min Act: -0.0000

Data Index: 1826668 (The Pile)

Max Activating Token Index: 101

Text #1

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1949. Min Act: -0.0000

Data Index: 1342334 (The Pile)

Max Activating Token Index: 417

Text #2

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.2332. Min Act: -0.0000

Data Index: 1308245 (The Pile)

Max Activating Token Index: 476

Text #3

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1761. Min Act: -0.0000

Data Index: 1367588 (The Pile)

Max Activating Token Index: 831

Text #4

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1850. Min Act: 0.0000

Data Index: 390781 (The Pile)

Max Activating Token Index: 547

Text #5

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1075. Min Act: -0.0000

Data Index: 1179760 (The Pile)

Max Activating Token Index: 932

Text #6

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1628. Min Act: -0.0000

Data Index: 932094 (The Pile)

Max Activating Token Index: 363

Text #7

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0751. Min Act: -0.0000

Data Index: 1577595 (The Pile)

Max Activating Token Index: 828

Text #8

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0329. Min Act: -0.0000

Data Index: 1801655 (The Pile)

Max Activating Token Index: 98

Text #9

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1279. Min Act: -0.0000

Data Index: 1602775 (The Pile)

Max Activating Token Index: 326

Text #10

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1051. Min Act: -0.0000

Data Index: 987650 (The Pile)

Max Activating Token Index: 434

Text #11

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0427. Min Act: -0.0000

Data Index: 1635000 (The Pile)

Max Activating Token Index: 275

Text #12

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0479. Min Act: -0.0000

Data Index: 1991735 (The Pile)

Max Activating Token Index: 269

Text #13

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.1362. Min Act: -0.0000

Data Index: 1985312 (The Pile)

Max Activating Token Index: 876

Text #14

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0049. Min Act: -0.0000

Data Index: 409031 (The Pile)

Max Activating Token Index: 844

Text #15

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0087. Min Act: -0.0000

Data Index: 1152532 (The Pile)

Max Activating Token Index: 129

Text #16

Max Range: 1.2332. Min Range: -1.2332

Max Act: 0.9946. Min Act: -0.0000

Data Index: 1799603 (The Pile)

Max Activating Token Index: 171

Text #17

Max Range: 1.2332. Min Range: -1.2332

Max Act: 1.0330. Min Act: -0.0000

Data Index: 237495 (The Pile)

Max Activating Token Index: 766

Text #18

Max Range: 1.2332. Min Range: -1.2332

Max Act: 0.9901. Min Act: -0.0000

Data Index: 1341104 (The Pile)

Max Activating Token Index: 754

Text #19

Max Range: 1.2332. Min Range: -1.2332

Max Act: 0.9536. Min Act: -0.0000

Data Index: 1365011 (The Pile)

Max Activating Token Index: 135