Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3895 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.4782. Min Act: -0.0000

Data Index: 134088 (The Pile)

Max Activating Token Index: 482

Text #1

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.4133. Min Act: -0.0000

Data Index: 19068 (The Pile)

Max Activating Token Index: 630

Text #2

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3387. Min Act: -0.0000

Data Index: 398370 (The Pile)

Max Activating Token Index: 990

Text #3

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3363. Min Act: -0.0000

Data Index: 750356 (The Pile)

Max Activating Token Index: 104

Text #4

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3185. Min Act: -0.0000

Data Index: 75559 (The Pile)

Max Activating Token Index: 604

Text #5

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2979. Min Act: -0.0000

Data Index: 629121 (The Pile)

Max Activating Token Index: 546

Text #6

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2998. Min Act: -0.0000

Data Index: 1215576 (The Pile)

Max Activating Token Index: 396

Text #7

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3294. Min Act: -0.0000

Data Index: 316038 (The Pile)

Max Activating Token Index: 148

Text #8

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2901. Min Act: -0.0000

Data Index: 1471471 (The Pile)

Max Activating Token Index: 138

Text #9

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3075. Min Act: -0.0000

Data Index: 1283642 (The Pile)

Max Activating Token Index: 139

Text #10

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2818. Min Act: -0.0000

Data Index: 402971 (The Pile)

Max Activating Token Index: 264

Text #11

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2916. Min Act: -0.0000

Data Index: 1015954 (The Pile)

Max Activating Token Index: 877

Text #12

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2915. Min Act: -0.0000

Data Index: 1588646 (The Pile)

Max Activating Token Index: 764

Text #13

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2845. Min Act: -0.0000

Data Index: 246526 (The Pile)

Max Activating Token Index: 151

Text #14

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3074. Min Act: -0.0000

Data Index: 492886 (The Pile)

Max Activating Token Index: 265

Text #15

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2922. Min Act: -0.0000

Data Index: 505363 (The Pile)

Max Activating Token Index: 444

Text #16

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3113. Min Act: -0.0000

Data Index: 1599849 (The Pile)

Max Activating Token Index: 545

Text #17

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2785. Min Act: -0.0000

Data Index: 913301 (The Pile)

Max Activating Token Index: 254

Text #18

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.2892. Min Act: -0.0000

Data Index: 1323857 (The Pile)

Max Activating Token Index: 241

Text #19

Max Range: 0.4782. Min Range: -0.4782

Max Act: 0.3128. Min Act: -0.0000

Data Index: 1822569 (The Pile)

Max Activating Token Index: 629