Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 878 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.4354. Min Act: -0.0000

Data Index: 1539948 (The Pile)

Max Activating Token Index: 196

Text #1

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.4490. Min Act: -0.0000

Data Index: 1025553 (The Pile)

Max Activating Token Index: 292

Text #2

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.4305. Min Act: -0.0000

Data Index: 994481 (The Pile)

Max Activating Token Index: 45

Text #3

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.2535. Min Act: -0.0000

Data Index: 1785905 (The Pile)

Max Activating Token Index: 81

Text #4

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.3729. Min Act: -0.0000

Data Index: 1819882 (The Pile)

Max Activating Token Index: 26

Text #5

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.2641. Min Act: -0.0000

Data Index: 442388 (The Pile)

Max Activating Token Index: 37

Text #6

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.1435. Min Act: -0.0000

Data Index: 1001667 (The Pile)

Max Activating Token Index: 650

Text #7

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.1282. Min Act: -0.0000

Data Index: 1341826 (The Pile)

Max Activating Token Index: 473

Text #8

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.2052. Min Act: -0.0000

Data Index: 405441 (The Pile)

Max Activating Token Index: 144

Text #9

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.1442. Min Act: -0.0000

Data Index: 1592373 (The Pile)

Max Activating Token Index: 252

Text #10

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.2170. Min Act: -0.0000

Data Index: 1609361 (The Pile)

Max Activating Token Index: 595

Text #11

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9916. Min Act: -0.0000

Data Index: 513654 (The Pile)

Max Activating Token Index: 421

Text #12

Max Range: 1.4490. Min Range: -1.4490

Max Act: 1.0737. Min Act: -0.0000

Data Index: 212057 (The Pile)

Max Activating Token Index: 774

Text #13

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9699. Min Act: -0.0000

Data Index: 1268119 (The Pile)

Max Activating Token Index: 843

Text #14

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9351. Min Act: -0.0000

Data Index: 1964683 (The Pile)

Max Activating Token Index: 524

Text #15

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9916. Min Act: -0.0000

Data Index: 234232 (The Pile)

Max Activating Token Index: 103

Text #16

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9797. Min Act: -0.0000

Data Index: 1646752 (The Pile)

Max Activating Token Index: 166

Text #17

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9760. Min Act: -0.0000

Data Index: 1178509 (The Pile)

Max Activating Token Index: 935

Text #18

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9463. Min Act: -0.0000

Data Index: 167514 (The Pile)

Max Activating Token Index: 79

Text #19

Max Range: 1.4490. Min Range: -1.4490

Max Act: 0.9516. Min Act: -0.0000

Data Index: 1742817 (The Pile)

Max Activating Token Index: 587