Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 499 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1074. Min Range: -1.1074

Max Act: 1.1074. Min Act: -0.0000

Data Index: 158667 (The Pile)

Max Activating Token Index: 529

Text #1

Max Range: 1.1074. Min Range: -1.1074

Max Act: 1.0228. Min Act: -0.0000

Data Index: 12099 (The Pile)

Max Activating Token Index: 421

Text #2

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.9433. Min Act: -0.0000

Data Index: 737092 (The Pile)

Max Activating Token Index: 178

Text #3

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8843. Min Act: -0.0000

Data Index: 264948 (The Pile)

Max Activating Token Index: 716

Text #4

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.9801. Min Act: -0.0000

Data Index: 1980001 (The Pile)

Max Activating Token Index: 880

Text #5

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8620. Min Act: -0.0000

Data Index: 209078 (The Pile)

Max Activating Token Index: 126

Text #6

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8291. Min Act: -0.0000

Data Index: 1518069 (The Pile)

Max Activating Token Index: 649

Text #7

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8303. Min Act: -0.0000

Data Index: 1870860 (The Pile)

Max Activating Token Index: 635

Text #8

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8293. Min Act: -0.0000

Data Index: 1348752 (The Pile)

Max Activating Token Index: 478

Text #9

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7801. Min Act: -0.0000

Data Index: 758150 (The Pile)

Max Activating Token Index: 505

Text #10

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8226. Min Act: -0.0000

Data Index: 39987 (The Pile)

Max Activating Token Index: 121

Text #11

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.8214. Min Act: -0.0000

Data Index: 111399 (The Pile)

Max Activating Token Index: 409

Text #12

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7697. Min Act: -0.0000

Data Index: 1822060 (The Pile)

Max Activating Token Index: 498

Text #13

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7981. Min Act: -0.0000

Data Index: 1056552 (The Pile)

Max Activating Token Index: 974

Text #14

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7375. Min Act: -0.0000

Data Index: 9649 (The Pile)

Max Activating Token Index: 848

Text #15

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.6929. Min Act: -0.0000

Data Index: 1939053 (The Pile)

Max Activating Token Index: 736

Text #16

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7658. Min Act: -0.0000

Data Index: 149494 (The Pile)

Max Activating Token Index: 856

Text #17

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.6898. Min Act: -0.0000

Data Index: 1076136 (The Pile)

Max Activating Token Index: 469

Text #18

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7101. Min Act: -0.0000

Data Index: 64352 (The Pile)

Max Activating Token Index: 726

Text #19

Max Range: 1.1074. Min Range: -1.1074

Max Act: 0.7217. Min Act: -0.0000

Data Index: 1052032 (The Pile)

Max Activating Token Index: 448