Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2733 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2104. Min Range: -1.2104

Max Act: 1.2104. Min Act: -0.0000

Data Index: 985545 (The Pile)

Max Activating Token Index: 225

Text #1

Max Range: 1.2104. Min Range: -1.2104

Max Act: 1.0843. Min Act: -0.0000

Data Index: 928559 (The Pile)

Max Activating Token Index: 211

Text #2

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.9090. Min Act: -0.0000

Data Index: 659979 (The Pile)

Max Activating Token Index: 263

Text #3

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.9720. Min Act: -0.0000

Data Index: 722454 (The Pile)

Max Activating Token Index: 90

Text #4

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.8315. Min Act: -0.0000

Data Index: 1145719 (The Pile)

Max Activating Token Index: 407

Text #5

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.8397. Min Act: -0.0000

Data Index: 1393138 (The Pile)

Max Activating Token Index: 582

Text #6

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7325. Min Act: -0.0000

Data Index: 1499844 (The Pile)

Max Activating Token Index: 969

Text #7

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.8964. Min Act: -0.0000

Data Index: 345730 (The Pile)

Max Activating Token Index: 892

Text #8

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7748. Min Act: -0.0000

Data Index: 738238 (The Pile)

Max Activating Token Index: 307

Text #9

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7321. Min Act: -0.0000

Data Index: 456044 (The Pile)

Max Activating Token Index: 907

Text #10

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7428. Min Act: -0.0000

Data Index: 618608 (The Pile)

Max Activating Token Index: 161

Text #11

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7430. Min Act: -0.0000

Data Index: 1771140 (The Pile)

Max Activating Token Index: 264

Text #12

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7196. Min Act: -0.0000

Data Index: 678903 (The Pile)

Max Activating Token Index: 439

Text #13

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.6945. Min Act: -0.0000

Data Index: 403720 (The Pile)

Max Activating Token Index: 602

Text #14

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7749. Min Act: -0.0000

Data Index: 30173 (The Pile)

Max Activating Token Index: 592

Text #15

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.6712. Min Act: -0.0000

Data Index: 443425 (The Pile)

Max Activating Token Index: 983

Text #16

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.6850. Min Act: -0.0000

Data Index: 1368319 (The Pile)

Max Activating Token Index: 585

Text #17

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.6590. Min Act: -0.0000

Data Index: 1633914 (The Pile)

Max Activating Token Index: 546

Text #18

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.7423. Min Act: -0.0000

Data Index: 1127196 (The Pile)

Max Activating Token Index: 783

Text #19

Max Range: 1.2104. Min Range: -1.2104

Max Act: 0.6948. Min Act: -0.0000

Data Index: 1157709 (The Pile)

Max Activating Token Index: 347